A Practical Guide for Policy Analysis:

The Eightfold Path to More Effective

Problern Solving

Eugene Bardach
Richard and Rhoda Goldman School of Publlc Policy
University of Calfornla at Berkeley

CHATHAM HOUSE PUBLlSHERS

SEVEN BRIDGBS PRESS, LLC

NEW YORK LONOON

Preface

TH15 15 A HANDBOOK of concepts and rncthods for working your way rhrough
a policy anaJysis. The presumed user is a beginning practirioner preparing to
underrake a policy analysis. 1 have developed the general approach and many of
the specific suggesrions over rwenry-five years of teaching poJicy analysis worlcshops
to first- and second-year graduare students at (he Richard and Rhoda Goldman
5chool of PubJic Policy, Universiry of California, Berkeley. 1 have also found this
handbook useful in teaching an undergraduate introducrion to public policy and
for executive education groups.
The handbook assurncs a familiarity with basic econornic concepts, including
those having to do wirh rnarket failures (including rnarker imperfections). Ir ls
nor meanc ro stand alone bur should be used in conjunccion wirh orher sources.
Five of thc best texrbooks in poJicy analysis, which amplify poinrs in chis handbook,
are

Edith Srokey and Richard Zeckhauser, A Prima flr Policy Analysis (New
York: Norron, 1978).
David L. Weimer and Aidan R. Vining, Policy Analysis: Concepts and
Practice (Upper Saddle River, N.J.: Prcntice Hall, 1999).
Roben D. Behn and James W. Vaupel, Quick Analysis for BuS) Decision-
Makers (NewYork: Basic Books, 1982).
Lee 5. Friedrnan, Microeconomic Policy Analysis (New York: McGraw-
r Hill, 1984)-revised edirion forrhcoming.
Duncan MacRae Jr., and Dale Whittingron, Expert Aduice for Policy
Choice: Analysis and Discourse (Washington, D.e.: Ceorgetown Univer-
sity Press, 1997).

1 wish to acknowledge the patience of and rhe helpful rcactions frorn all the
students and fric:nds who .havc made use of this handbook, especially rhose who
pue up wirh its earlier versions. Special thanks are due to Roben Behn, Sandford
xlv A PRACTICAL GUIDE FOR POL/CY ANALYSIS INTRODUCTlON xv

pracrice and ro srudents and others who, for whatevcr reasons, are arte:mpting to
lo ole ar rhe world rhrough rhe: e:ye:sof a practitioner.
Policy Analysis: More Art than Science
Policy analysis is more art rhan scie:nce. Ir draws on inruition as much as method,
Nevertheless, give:n [he: choice: berween advice mar irnposes too rnuch structure
on rhe problern-solving proce:ss or roo lirrle, rnost be:ginning practitioners quire
reasonably pre:fer roo mucho 1 have: rherefore developed an approach 1 call me
Eightfold Parh. The primary utiliry of chis structured approach is rhar ir reminds
you of irnporranr rasks and choice:s that otherwise mighr slip your rnind; its
primary drawback is that, rakcn by itself ir can be mechanistic.
he Eightfold Path
Define: thc Problem
Assernble Some Evidence
The spirit in which you takc any orie of thesc sre:ps, cspecially in me: carlicst
phases of your projecr, should be highly renrative. As you move: rhrough [he
problern-solving process, you will probably keep changing your problem defini-
tion, your menu of alrernarives, your ser of evaluative criteria, your sense of what
evidencc bears' on the problern, and so on. Wirh each successivc iceration you
will become a bit more confident that you are: on [he: righr track, rhat you are
focusing on rhe right question, and so on. This can be a frusrrating process, bur
ir can also be re:waeding-provide:d you can leaen to enjoy rhe challe:nge of search,
discovery, and inve:ncion.
Some of the Guldelines Are Practlcal, but Most Are Conceptual
Mosr of rhe conceprs use:d will seern obvious, However, thcre are exceprions,
Fi~sr. rechnical rerrns are sornctimes employed. Sccond, sornc commonsense: tcrrns
may be used in a special way that strips rhern of cerrain connotations and perhaps
irnports others. For the rnosr part, all these concepts will bccorne inrelligible
~hrough experience and practice,

Consrruct rhe Alternarives

Selecr the Crireria

The Concepts Come Embedded in Concrete Partlculars
Projecr the Ourcomc:s In real life, policy problcrns appear as a confusing welter of de:tails: personaliries,
Confront [he Trade-Offs interest groups, rhetorical dernands, budget figures, legal rules and inrerpretarions,
bure:aucraric routines, cirizcn attirudcs, and so on. Yet rhe concepts describcd in
Decide!
chis handbook are formulate:d in the abstracto You therefore need to lcarn [Q "sec"
Te:ll Your Story the analytic concepcs in rhe concrete rnanifestarions. of eve:ryday lfe:.

These: sreps are nor ne:cessarily rakcn in precisely rhs order, nor are all of
thern necessarily significant in every problem. Howe:ver, an dforr to define me:
proble:m is usually che righe srarting place, and te:Uing thestory is almosr inevitably
thc e:nding poinr, Consrruceng alternatives and selecting crircria for evaluating
thcm musc surely come roward rh.beginning of the process. Assembling so me
evidence is actually a step rhar recurs chroughour the entire process, and ir applic:s
"
parricularly ro efforrs to define [he problem and ro project the outcornes of rhe
alternarives being considered.
Iteration 15 Continual
The proble:m-solving proce:ss-be:ing a proce:ss of erial and error-is iteracive, so
rhat you usually must repear e:ach of rhese sre:ps, sometimos more rhan once.
Your Final Product
So whar will your final produce look like? He:re is a very rough sketch of a typical
'wrltren policy-analytic re:pore: In a coherenr narrative sryle you will describe some
problern that needs to be mitigaeed or solved. You will lay out a few alternarive
courses of aceon that rnighr be taken, To each course of acron you wil! atrach
a set of projected outcomes that you think your client or audience would care
abour, suggesting the cvidentiary grounds for your projections. If no alrernative
dominates all orher aleernatves wirh re:spece [Q all rhe evaluarive crircria of intcrcsr,
you will indicare me nature and magnitude of me rrade-ofls implicit in differcnc
policy choices. Depending on rhe clicnr's expccrations, you might state your own
recornmendarion as to which alrernative should be chosen.
 I
A PRACTICAL GUIDE FOR POLlCY ANALYSIS tj

e Spirit of the Eightfold Path


e spirit of me: Eightfold Path is, 1 hope, an economizing and uplifting spiric,
alyzing public policy problerns is a cornplex activiry. Ir is casy ro get los e, wasre
t
.r of rime, bccorne demoralized. Other manuals and rextbooks in polcy analysis
very concerned thar you gcr the analysis "righr," in sornc sense. I hope chis

: will hclp in that rcspecr roo. Bur, even more, I hope that this one'will help
l get ir done wirh reasonable efficiency as wcll. .' :
I PART r
The Eightfold Path

1. Define the Problern

YOUR FIRST PROBLEM DEFINITION is a crucial step. Ir gives you (1) a
reason for doing ;:11 me work necessary to complete me projecr and (2.) a sense
of dirccrion foe youe evidcncc-gathcring acriviry. And in the last phases of me:
policy analysis, youe final problein definition will probably help you strucrurc
how you tel! your Itory.
Usually, the raw material for your inicial problem definirion comes from
your clie:ne and derives from the .ordinary language: of debate and discussion in
me: client's political environrnent, language I call generically HUt: rhetoric. This
rhetoric may be narrowly confined to a seemingly technical problem or broadly
located in a conerove:rsy of wide: social inreresr. In eirher case, you have to get
bcncath rhe rhetoric to define: a problem mar is analyrically manage:able and rhat
rnakes scnse in lighr of me political and institutional means available for mitigar-
ing ir.
Use: the raw material of issue rheroric wirh careo Ir ofte:n oinrs to sorne
condition of e wor at people don r i e or considcr "bad" in so me sense, iK'e:
"fc:enage pregnancy," "medIa vlOlence," or" lobal warmin ." The:se evalua~
not necessari y nce (O be takcn at face value. You will sornerimes wish to explore:
the philosophical and ernpirical grounds on which you, your client, or othcrs in
your eventual audiencc should or should nor considcr the allcgcd condirion
"bad." Furthermore, issue rhe(Qri~ may peine (Q somc allcge:d-bur not neccssarily
real-cause of thc rroubling condition, for exarnple, "welfare" or "hurnan wasrcful-
ness." You wanr nor simply to echo' the issue rhetoric in your problcm dcfinirion,
bur [Q use ir as raw material for a provisional problc:m definron mar you hope
will prove analycically useful. .
Some: issues may connote more: than one problcm. Depe:nding on the audi-
cnce, for exarnplc, "reenage pregnancy" might connote sexual immorality, rhe
blighcing of young pcoplc's and thcir childrcri's life: chanccs, exploirarion of
 2
A PRACTICAL GUIDE FOR POllCY ANALYSIS
THE EIGHTFOLD PATH 3

raxpayers, and social disinregrari~n. Usually you will wanr to de:re:r~e: a primary
warranr definirian as ,public roblems and r e eb le itimatd ra~~ for
problem focus; o therwise you will find mar' me analysis ge:rs out of hand. But if
amelioranon y pu IC resources?";-{r is usually hdpful to view me situaticn
rhe: proble:ms areri'r toa complcared, you may fe:d you are: willing to define more
rhan one. ' rhrough clle market [ailur lens.' In' irs simplest formularon, rnarket failure occurs
when the tecbnical properties af a good or service

8 malee ir hard to collecr payment from all the potential bcndl~iaries-for

Thlnk o] Deflclts and Excesses
instancc, the large nurnber of people who profir, albelr indirecdy, from
Ir ofren-bur nor always=-helps [O think In terrns o f d cncir
". an d excess. For advances in basic science:
insrance: make ir hard to collect fram the beneficiaries of consumption the (fue
economic COS[ of making use of the good or service-fresh air thar vehi-
"There are toa many horneless pea pIe in me: Uni~ed Stares." cle owners use:'as a sink for thcir auto emissions;
"The d~mand for agriculrural water is gro~ing faster rhan our abiliry ro rnake ir hard far consumers (and sornetimes suppliers) to know me true
supply Ir at an acccprable financial and environrnental cosr." qualiries of the good or service mey are acquiring-many repair-rype ser-
"California's popularion of school-age children is gr?:":,,i.~gat 4,000 per vices, including those performed by physicians as well as those performed
year, and our abiliry to develop the physical facilities in which to educare: by 'auto mcchanics;
rhern 15 nor growing nearly as fasr." malee me cost of praducing me marginal unir lower than me average cosr
wirhin me relevant range of demand-a magazine article distribured via
It afeen htlps to inclt '00 in rhe definition (e g "ro bi "U e4e Internet.
all " . .., o Ig, roa
~m '. ~,rowln~ toa slowly,' "growing roo fasr"). iese Iast rw~ phrases (abour
g:o':Jng ) ,;emmd us rhat problems dese:rving our attenrion don'r necessarily Ir is hard to overestimare rhe irnporrance of rhis peine, for in mose-rhough
exlS[. roday buc are (ar easr porcnrially) in prospect for rhe future, whe:mcr ncar nor all':"-siruations where no actual market failures can be idcntified, people's
or disranr, privaretroubles cannot rypically be arneliorared by even the most well-inrcnrioned
Hov.:eve:r, it doe:s nor heJp ro rhink in rerrns of deficir andexcess when your governrnenral inrervcntions. And even whcn so me amelioracion is possible, rhcre
probl~m 15 a~ alr~ady ,:ell-structured decision problern, 'for example, "Dump the are usually many advcrse side effecrs. In so me cases, it may nevertheless be
drc:dg.mg spo.lis eirher In me Bay or sorncwhere our in the Pacific Occan." Nor worthwhile ro pay rhe price of rhcse side effecrs, but such calcularions musr be
do~s I~ hdp If your challenge is ro invenr any way ro accomplish sorne defined done carefully and scrupulausly.
objcctive, far cxarnple ."Fin~ sorne granr funds ro clase the anticipated gap berween Besides markct failures, me main situations whcre privatc troublcs can warrant
revenues and expendirures. The:se: decision- and invention-rypc "problcrns" are deflnition as public problcms are
?robl.ems Jo.r me policy analysr bur are nor the: son of problerns I am addrcssing
In mIS secClon. ' , brakdowns of sysrerns, such as family relationships, that occur largcly
outside rnarkcts:
, me concern of many cirizens abour low living srandards thar arise pre-
The Deflnltlof/ Should Be Evaluati~e .cisely because markcrs function well and do nor reward individuals very
! cnerously if rhey lack rnarkerable assers
Rcme:n:be:r rhe: de~ of a "problern" usually means thar people think mece is
somerhmg wron.g with rhe world. Bur note -char wrong is a very debarable terrn.
Nor everyone will rhi?k rhat the faces you (or orhers) 'have dcfine:d as a p~ l. See chapea 5 in David 1. Weimou and Aidan R. Vining, Policy Analysis: ConceplI and Prac-
afeiattyTpro~1U. lar eacpetson may aPJrly a difLcllt evafuatIve framcwork tice (Upper Saddle Rver, N.J.: Prentlce Hall. 1999), For a persuasve analysis of rnosr rradi-
tional marker failures in transaction cose rerrns, sce Richard O. Zerbe Jc., and Howard E.
t.!l-: aes . ceso , n ortunate y, e en: are no o ous or accepre ways ro resolve McCucdy. "The Failure of Marker Failure," Jo~mal o/ Policy AnalJII and ManoKmunr 1B,
phliosophlcal dfferc:nces of chis rype. no. 4 (1999): 55B-7B. Zerbe and MeCurdy also emphasiae the rich variery of inrerven-
A common philosophical as well as practica! quesricn is "~ar privaee trotiliks rions besides [hose underraken by governrnenr ro rernedy traditionally conceived "rnarker
f"ilures, '
 A PRACTICAL GUIDE FDR POLlCY ANALYSIS THE EIGHTFOLD PATH 5

the existence of discrirninarion against racial and orher rninorities: necdcd scrutiny. Scc "Projecr rhe Ourcorncs" for
definitions can easi 1y escape
che failurc of governmenr to function wdl in arcas whcrc Ir is tradition- further discussion.
a1ly expecred to act eflectively kg., in providing public schools).

MIssing an Opportunity Is a Problem

Intify Ir Possible
definirion should, insofar as possible, include a quantitative feature. Assertions
eficit or excess should come with magnitudes attached, How bjg is "roo big"?
'{ small is "toO small"? How abour "too slowl "1 How &ue "roo fast"? In
examp es above, how many orne ess people are rhere in me United Statesi
v many acre-fea of water are used now, and how dces that arnount compare
I the dernand in sorne specified fumre year (given certain assumptions about
:r pricing)? Exactly what is "our abiliry to develop physical facilities," and
r do we expc:ct le to graw, or shrink, ovcr time? ' prafessionals is ser by complaints, rhrears, worries, and rroublcs. There IS ofrcn
If necessary, gamer information to help you calibrate me relevant magnitudes.
rhe discussion under "Assernble Some Evidence."
In many or rnost cases, you wiU have to cstimare-c-or "guessrimare," more
Iy-rhe magnitudes in question. Sornetimes you should furnish a rang as well
point esrirnate of magnitudes (e.g., "Our bese guess of ehe number of horneless
ions in families is 250,000, a1ehough the truth could lie berween 100,000 and
.ooo").
special case of "a problern" is a~ opporjunity misscd, Is ir nor ra~her .small-
minded to rhink of policy analysis as devotc:d rnerely to me ameho~a[lo~. of
"problerns"? May policy analysis not rise abov~. rhe redious :m
d ~nmsplflng
business of patching and fixing? Can we not aspire (Q a world I~ w~l;h "" ~~
idencify opporruniries to do creative-nor to s~y won~erful-mmgs .. lf Ir ain e
broke don't flx it" is a confining idea, and certainly policy analysts, pohcymakers,
and p'ublic managers ought nor to allow the "problem': focus to restrict rhc sca~ch
for plausible: opporrunities. Unforrunately, the .worlung agenda of mos~ policy
little rime or energy lcft ovcr to rhink about impraveme~ts that no .one has
Identified as missing. Still, f larent opportunities are really IYlllg around, it would
be a pity to ignore thern. .' . ,
Where do we find opporruniries for creauve pollcy lmprove:men~ rhat hav~n t
first been identified by complaints, threats, and so on? Linle academl~ or te~hnlcal
theory is available to answcr mis question. But Box l.I (p. 6) contams a llst thar
is suggcstvc:.

rdltlons That Cause Problems Are Also Problems Common Pitfalls in problem Depnitlon-
(
le problem conditions are not experienced as troublesorne per se by citizens Problern definirion is a deceptively simple srep. Ir is a step bcset by ac least rwo
are perceived by rhern, or by analysts working on their behalf ro be causes dangeraus pitfa.lls. . .
roubles, Ir is sornerimes useful to diagnose one or more a1leged causes of this Defining the solutlon Into the "problem." Your problern definidon should
: and to define these as problema to be mitigarcd or removed, forinstance, not include an implicit solution intraduced by sernantic carelessness. Proj~~(ed
le of rhe problerns in rhe air pollurion arca is rhat srates have not been willing solutions rnust be evaluatcd empirically and not legirirnated rnerely by definicion.
orce rnotorists to keep rheir engines runed up and rheir exhaust systerns in Therefore, keep the prablc:m definition srripped down (O a rnere description, and
rer order." cavc open where you willlook for solutons.
Note mar rhis sort of problern deflnition is not merdy descriprive but is a1so
n ostic. It implicidy asserts that so me condition, which may or may nor be Don', say: "The:re is (00 lird~ sheltcr for horncless fanlilies." This fo~mula-

bling to people per se, is an irnporranr cause of 50 me orher condirion mar is
.ed rroubling. Problem definirions that pretend to sornc diagnostic power can
rseful, but they can a1so be rreacherous. Suppose, afrer a11, rhat me causal
nosis is mistaken or mislc:ading, for exarnple, that states' unwillingness to
rce engine rnainrenance routines is not in fact a very important cause of air
ution, Becausc definition in sornc conrexrs connotes legitimate arbitraririess
1define '[ustice' to rnean ... ") the causal.!daims irnplicir in diagnoscic problem
don mighe inadverrenrly imply rhat "more shelter" is rhe bese so~~tlon
and rnight inhibir you fram thinking about ways (Q prevent [amilics
fram becoming homeless in rhe, firsr place. Try instead: "There are (00
many horneless familie:s." ".'
Don', say: "New schools are being builr too slowly. T~lS ~o:mulatlon
could imply "more schools" as the solucion and could l.nhlb1Cyou :rom
rhinking abour ways [O use existing facilities more efficlently. Try InItead
 THE EIGHTFOLD PATH 7
6 A PRACTICAL GUIDE FOR POLiCY ANALYSIS

"There are tOO many schoolchildren relative tO rhe currendy available

BOX 1.1. SDME GENERIC OPPORTUNITIES' fOR SOCIAL IMPROVEMENT THAT
classroom space."
OFTEN GO UNNOTICED
A tpoff rhar you' re probably smuggling an implicit soluron into me problem .
By-productt o/ personal aspirations. Ir is possible ro srcucrure new i .
or creare ..
ne,,:, opp~rrunltles
fi ncennves
or personal advantage or sarsfacrion mar can indi- , definicion is to hear your~df saying, "Aha, but rhat' s not me real problem; the
l
recdy r~sulr I~ SOCial ~enefir (e.g., offering to share rhe benefirs of cosr-reducing ".!: real problcm is . :." While there are berter and worse ways {O conceptuali1.c a
~'.
innovauons with public sector ernployees who conceive them and lm pl problem, or (O solve a problem, ic srrecches ordinary usage (00 much (O say rhat
rhern). emenr
:~ one problem could be "more (less) real" than anorher.
_L Complementarisy. Two or more activiries can be joined so mar each ak Be skeptlcal abaut the causal daims Impllclt In diagnastc prablem defl-
mee othcr more producriv (e.g., pu bliIC wor l'es consrrucnon and combaring m es nltlons. 1 said above rhar "condicions that cause problems are also problems."
p oymenr). unern-
l Howev~r, me causesmust be real, not merely assumed. You have to evalate rhe
Development. A sequence of activiries or operarions may have me orenrial causal that gacs from thc situaran itsclf {O me
chain bad rhings it is alleged {O
to be ~rranged to take advanrage of a developrncnral process (e p. I cause, ~nd to convincc yourself rhat rhe causal chain is real. For instance, for
Iare I f I bili ,g., asseSSlng we -
~ ients or emp oya. iury and vocarional inrerest before, rather than afrer
:0.:
so me pcople, "cocaine use" is not a problem in itsdf, but ir might be a problcm
sending rhern out on a [ob search). '
'11. if ir leads to crirne, poor health, family disintegration, and 50 on. But does ir lead
There are unrealiaed possibilties for exchange mar would increase
'a1Exalchangwe.
(O rhese outcomes, .and to what degrec? The evidencc on mis question sho.uld be
SOCI
1"1 v ue . e rypi
'/ ICall y desi
cSlgn po l'ICJes
. to exploir rhose that 'simulare rnarker-
~ (e arrangemencs (e.g., poll~rion permit aucrions, and arrangemenrs to reirn- evaluated very carcfully before you decide it's okay {O work wirh a p~oblein
urse an a.gerrcy fo~ services ir renders anothcr agency's clienrs or cusrorners) " definiEon' involving "(00 much cocaine use."
lv.fi1lrtplefimclton!. A sysrem can be designed so that one fearure can be' used
ro t:per orm rwo or. more funcr ons (e,g., w h en a rax adminisrrator dramarizes an
cnrorcernenr, case in such a way as b OUl _L to d erer porenrial
. violarors and to reas-
iterate
sure nonvlOla~~rs thar mer are not being rnade inro sucleers for their hones ). Problem definiron is a crucial scep. Bur becausc ir is hard {O ger ir righr, you
Nontradisional part/CIpanlI. Line-level ernployees of public . n ty might take that sarne step again and again. Over me coursc of your analyric work,
have knowled f . agencies o en
. ge o potcntial program improvemenrs mar could useful!y b .
porated I h ., l" e incor- your empirical and conceptual understanding will evolve. Alsa, as you begin (O
. ,. neo t e agencies po icies and operarions . The same
. I'S r.
yUe o f me
_L agen-
rule out alrernarve approaches to solving or mitigaring your problem, you will
eres cus~ome:s o~ clienrs or the parries rhat they regula re. '
probably want to sculpr me problem definicion so mar, in rhe end, you and me
Ra~/onal12:at/on. Purely rechnical rationalizarion of a sysrem is ossibl
polirical system wil1 have sornc chance of anacking the problem successfully.
sh~~r.~lngfi que~es by deliberare spacing of arrival times, Q[ crearin: contr:c~~'~~
so I I In orn: agreements rhar are vulnerable ro decay and misundersrandin ) Fina1lYJ if you are working in an office or agcncy contcxt, you wil! implicidy be
~m~agmg. By rummaging menrally, one rnighr discover novel uses in g. negorating a mucually acceprablc pioblem defmition with your analyst calleagues
1
seerningly Improbable bur readily available marerials (e g using _L
Ule automo bil e
'J
and your hierarchical supe:riors.
reglsrUratl~n sysrem as a vehicle for carrying out vorer registraran as wel!)
. 'nderutilized
. . capncity . An examp l'e, In many communmes,. . is schoo! . facili-
;'Ies rh~r are utilized for relarively lim~red purposes for only pare of me day and 2. Assemble Some Evldence
or. on yhi parr of ..the year-alrhough school official. would be qU1C. k ro warn _L Ular All of your time doing a policy analysis is spent in rwo a~Eyi(ies: __thinking
U'lng t 1. capaclty withour harming schoo! funcrions 15'nor alway' easy.
(sometlmes aloud and somerlmes with others)- and husding dara that can-----:
be

2. Some analysrs also clairn that ir is simply nor wormwhile ea define as "probltms condi-
tions rhar cannor be amelior:ued: Problems are berree ueared as oppoeeuniries foe improve-
mef\ti dcfined problems, as peablems of choice berween alrernarive me.OS ea eealiu: a given
opporruni'Y. Thc proccss of problcm definiran would rhen be one of search, crearion, .nd
inltial ex.min.rion of ideas foe salurion unril problem of choice is reachcd: O.vid Oery.
Problem Dtfinilion in Policy .nalyit (L.wrtncc: Universi'Y Pross of Kan; , 1984). 27.

A PRACTICAL GUIDE FOR POLlCY ANALYSIS
:n~' .dcnce Of these rwo activiries, thinking is by far the more important,
t husding data ta es much more time: reading documents, huncing in libraries,
ring over studies and sratisccs, intervie:wing people, traveling to interviews and
jring for appoinrrnenrs, and so on.
The real-world setrings in which policy analysis is done rarcly afford the rime:
. a re:se:arch effoft that would please a carc:ful academic re:se:arche:r.In fa~r, time:
.ssure is probably almost as dange:rous an e:ne:my of high-quality policy analysis
polirically rnorivated bias, if not more: so. The:rc:fore, ir is essential ro econornize
your data collection activiries, The ke:y to -cconornizing is this: Ter
to colle:ct
1 rhose: dara mat can be: turned ineo "informaron" that in rurn, c n be:
1Ve:rted inro "evi ence: t at as so me: bearing on your prob,km. ; anyrhing you actually carc about, Pco le: often do this be:cause: runnin' around
-Por the 10gicaIly mindcd, here are: sorne definitions: Data are facts-or, some:
ght say, representations of facrs-abour (he warld. Data indude: all sorts of
tistics bur go we:1Ibc:yond sraristics, too. Data also include, for instance, facrs
cut an age:ncy manager' s ability ro deal cdnstructively with the prc:ss. Information
data that has "rneaning," in (he sense rhar ir can help you sort the world into
ferent logical or e:mpirical categories. The: pre:vale:nce: of cigare:tte: smoking in
e: diffe:renc counrries is data. bur these data be:come: inforrnation whe:n you
:ide: it is inreresting ro array the countries compararively (e.g., from lowesr to
;hest prcvalcnce}. Euidence is information rhat affects the cxisting ~c1ie:fs of
portant people (including yoursdf) about significant Features of [he problem
u are studying and how ir rnighc be: solved or rnirigared. Differenrial prevalencc
smoking, for instance, can becorne evidence be:aring on hypothc:sc:s about .
fe:re:neial levels of conce:rn abour personal health across countries, I
You need cvidence forthree principal purposes. One purpose is toasscss the
:un: and extenc of rhe problem(s) you are r in to define:. A secon~
: partlcular fe:arures o e:concrete: po i sitUarion you are: e:n ed in stud in .
0nsrance, y to ow-or guc:ss-about agency workloads, recen'[
dge:t figures, dernographic changc:s 'in a service area, the political ideology of
: agency chief rhe competency of rhe middle-Ievd managers in the agency,
:l me: current attirudcs of so me: other age:ncy rhat nominally coopc:rare:s wirh
s one on sornc problern. The third purpose is .so assess policie:s that have: been
,ught, by at leasr some people, to have: workcd cffecrivc:ly in situations appare:ndy
ular to your own, in orhe:r jurisdictions, perh;p5, or ar other ti~.l,All thre:~
[.poses are: rdevant to die: goa1 of produLng le~oje:ction;-of possible
.ley outcornes.
Because each of thesc purpose:s becomes salienr in differcnr phasc:s of the
Sometimos rhese siruacioru will have becn evaluarcd srarlstically, and somerirnes
Parr IIl. "Sm.r< (Bese) Praccices' Research."
THE EIGHTFOLD PATH 9
policy analysis proce:ss. this "Assernble Some Evidence" step on th~ Eightfold
Path will be taken more: [han once:, but wirh a different focus each umc.
, I
Think 8efore You Collect
Thinking and colle:cting data are cornplernenrary activities: you can be: a much
more: efficienr collector of data if you think, and keep on thinking, abour whar you
do and don't need (or-want) to know, and why. The principal-and e:xce:edingly
common-mistake: rnade by be:ginne:rs and veterans alike: is to spe:nd rime: colle:cting
data thar have linle or no porential ro be: developed into evidcnce concerning
collectin ' a ooks and fe:c:Is roductive: whe:reas Irst-rate: inking is hard and
fruscrating. Also, me: people paying for your work rend to. be: re:~sure ~t, w en
they sce you busily collccting data. somehow they are: ge:mng their mone:y s worth.
The value of evldence. Since most evidcncc is cosdy to produce, you rnust
weigh its likdy cost against its likdy value. How is its likdy value to be: estimacedi
The: answer may be: casr in a declsion-analytic frame:work (de:cision crees), though
you should reme:mber thar me proce:ss of making a decisin involve:s a great many
elernenrs prior to the: rnorncnt of actual choice, such as defining a useful problem,
thinking up better candidate: solurions, and selecting a usc:ful model. In general,
me: value: of any pie:ce of evidence depcnds on
me likc:lihood thar ir will cause: you to subsritute sorne beuer decisin for
whatever decision you would have rnade wirhout it (which might have
been an "acceptable" decisin in and of itself):
me: like:lihood rhar the substituted decisin will, dire:ctly or indire:ccly, pro-
duce: a better policy outcorne than the outcorne that would have becn
produced by the original decisin;
me: magnitude of the difference in value: berween the likely-ro-be im-
prove:d ourcorne and me original outcome:.
Self-control. Ir is surprising how well you can do in many cases by garhering
no evidcnce at all bur simply sirring down and rhlnking sornerhing through and
then making sorne serious educated guessc:s. There is nothing shame:ful about
acting on such gue:sstimatc:s and therehy conserving your dara-collecring time: and
e:nergic:s for answe:ring quesrions for which good evidence is re:ally ne:cc:ssary. See
Pare II, "Gathering Data for Policy Research."
A helpful check on yoursc:l{ to pn:ve:nt yoursclf from collecring useless data.
is to ask yourself the following quesrions be:fore:embarking on some: data colle:crion
no" see
' venture:
 -
10 A PRACTICAl GUIDE FOR POllCY ANAlYSIS
"SUppOSC the data rurn OUt ro loole like so-and-so as opposed ro thus-
and-such. What irnplication would chat-have for my undcrStanding of
how ro solve this problem?"
"Cornpared ro my ?est guess .about how the data willlook once I've goe
thcrn, how much diffcrcnr rnighr rhey look if 1 acrually took the trouble
ro gee rhern?"
"How much is ir worrh ro me ro confirm -the actual di.fFercnce berween
what 1 c~~ guess and whar 1 can learn about me world by really getting
rhe data?
Ir is this son of crirical attitude abour me value of expensive data collection
that often leads good and experienced policy analysts to make: do with back-of-
the-e~velope. esrirnares. However. none of rhe above is meanc to be an excuse for
shlr~(Jng the job of getting good data-and somctirnes ots of thern, at huge costs
In nrne and money-.when yo~'ve convinced yourself rhac the investment really
wo.uldpay off. There: s an obvious and crirical diffen:nce berween jusrifiable and
unjustifiable guesstimares.
Do a Literature Review
There is hard.ly a problern ~irhoue some acadernic discipline or professional
associannn dOlng r~search o~ its causes and solurioris. le is easy to find journals
and :vanous professional publJcauons disseminating research resulrs, theories, case
srudies, rhe .musings of experienced pracririoriers, and so on. The Internet brings
much of rhis to your deskrop, buc some of the bese vehicles are bctter accessed
by browsing the periodical shelves of universiry or government libraries.
Advocacy. orga~i2ation.s oft~n publish agrear dcal of inceresting work and
may take special pams to disserniriare their work on the Inrerner, However, there
IS a danger of rdying too much on such sources jusr because they are readily
avatlable.
Survey "Best Practlce" !.
Th~ chances are that rhe problem you are srudying is not unique, and rhat
policyrnakers and public managers in orher jurisdictions, perhaps not very different
from (he one you are srudying. have dealr wirh ir in sorne fashion. See if you
can track down some of rhese pasr solurions and see if you can extrapolare thern
ro the situanon you are srudying. The exrrapolation process is complicared,
thou~h. See Pare Il l, 'Srnarr (Bese) Pracrices' Research. Understanding and
Making Use of \'(fhat Look like Good Ideas from Somewhere Else."
THE EIGHTFOLD PATH 11
..
Use Analogies
Somedmes' ir pays to gathe:r data about things rhar are, on the surface, quite:
unlike rhe problern you are studying but thar, under che surface, show insrructive
similarities. For instance, your understanding of how a rnerit pay plan for compen-
sating managers in the public sector might work could perhaps be improved by
seeing how similar schernes work in me privare sector. Or if you are working on
the problem of how a stare can discipline, and perhaps disbar, incornperenr
artorneys. you might usefully sperid a good deal of your rime learning abour how
the me:dical profession handles problems of physician incornperence. If you are
working.on how to reduce neighborhoods' resistance to accepring low-incorne
housing projecrs, you could usefully look inro me lirerature on communiry resis-
tance ro accepting solid-waste incinerators.
As rhese exarnples suggest. so me analogies are easicr to peroeive, and te malee
scnse or.' than cchers. le takes a lirtle imaginarion te see insrrucrive analogies .and,
occasionaliy. a little daring to rey ro convince others to see both me usefulness
of the analogy and its inevitable limitations.
Start Early
You are: ofren dependent on rhe busy schedules of other very busy people whom
you ask te furnish you wirh inforrnation or opporrunities for interviews, Ir is
extrernely important ro pur in requests for inforrnarion, especially interviews, well
in advance of when you expece to wanr te have complcred rhc data co11ecrion.4
Touching Base, Gaining Credibi/lty, Brokering Consensus
The process of assembling evidence inevieably has a polirical as well as a purely
analyrical purpose. Sometimes ir entails touching base wirh potencial critics of
your work so that they will not be able: to complain that you ignored their
perspecrives. By rnaking yourself known to potential supporrers of your work,
you mar also be able: to crea te a cadre: of defenders,
A. more cornplex objective, where appropriate, mighr be to blend policy
analysis with the process of improvlng a policy idea or decisin duiing the course
of implemenration. (See the following discussion of"lmprovabllicy" as a crirerion.)
This entails obraining "feedback" from participants, usually in an iterarive process,
4. For a useful descriprion of how co conduce literarure reviews, lib ra ry searches, phone inrer-
views nd personal inrerviews, see Weimer and Vining. Palicy Anaiysis, chap. IDA. See also
Part Il, "Gathering Data for Policy Research."
12 A PRACTICAL GUIDE FOR POLlCY ANALYSIS
THE EIGHTFOLD PATH 13

and sharing sornc of your own reactions with thern. You becorne more of a
partner in the process rhan an ourside observer and diagnostician-An even more
cornplex and challenging role is for you to becorne a particular type of "parrner,"
a faciliraror and broker, whether by acting as a transmission belt from one person
to another or by convening meetings and other garherings. ! One way to coax your creaciviry is to refer ro che checklist in Appendix
freeng the Captlve Mnd
In exchange for access to data and a ready-made worldview, researchers sornetirnes
uncrirically accept problem definitions and preferred solutions from kindly infor-
manes (not to rnention from paying clients or employers). To' counter such
ternprations, be sure to make contact with individuals or factions whom you would
expecr to disagree-the more sharply the better=wirh your kindly informants. A
time-saving, but only parcial, substituto is to ask your kindly informant, "Who
might objecr strongly to your point of view about this, and why might they
do so?"
3. Construct the Alternatives
Try to inuent alternarives rhat rnight prove to be superior to the alterna-
ti ves currently being discussed by the key political actors. It's good to
brainsrorrn, to try ro be crcative=-but don't expect that you'll necessarily
produce much berter ideas than other people have already thought abour,
A, "ThingsGovernments Do." For each entry on the lisr, ask yoursdf,
"Might ir make sense to rry so me version of this generic srraregy ro help
mirigate this problerni'" Because ir is a long and cornprehensive list, the
answer with n:spect to any single srrategy will usually be negative. Going
through the list systemarically is word~hile, however. Because the list
is not very long, with experience you willneed to spend only a few
minutes to decide whether any ideas there rnighr be worth considering
further.
A1ways include in your firsr approach tothe problern the alternative "Ler
present trends continue undisturbed." You nccd to do this because che
world is full of natura!ly occurring change, and so me of these ongoing
changes mighr mirigare the problem on which you are working. (Nore
that 1 am not characcerizing this alrernarive as "Do Norhing.' Ir is not
possible to "do norhing.' Mosr of the trends in rnotion will probably per-
sist and alter the problern, whether for better or for worse.)

By alternativa 1 mean something like "policy oprions," or "alternarive courses

In rnost cases, however, chis "let-present-rrends-continue" oprion will drop
of action," or "alrernative srrategies of inrervenrion to solve or rnitigate the
problern." out of your final analysis, This happens because, if you do your problem definirion
work well, you will end up wirh an imporrant problem in your sighrs thar in
most cases can be mirigared to sorne degre by sorne affirrnative acciono
.Inspect the mosr common sources of "natural" change in the public policy
Start Comptehenslve, End Up focused environrnenr to see if any will affect me scope of rhe problcm:
In the last stagesof your analysis, you won't want ro be assessing more rhan thrce
or four principal alternatives, BU[ in the beginning, you should err on the side 1. Political changes followingelections, as well as changes induced by the
of comprehensiveness: Make a list of all the alrernatives you might wish ro consider prospecrs of having to conrest an elcction
in the course of your analysis. Later on you will discard so me obvious losers, 2. Changes in unemployment and inflation rates that accompany the busi-
combine others, and reorganize srill others into a single "basic" alternative with ness cycle
one or more subsidiary "variants." For your initial lisr, rhough, where should you 3. The changing "righrness" _or "looseness" of agency budgets caused by
rurn for ideas?
I overall taxing and spending policies
4. Demographic changes, such as popularion migration patterns and popula-
Note the alternatives that key politica! actors are acrively proposing or tion "bulges" moving through certain ages.
seern to have on their minds . .These may include pecple's per ideas, insri-
rutions' inventories of "off-rhe-shelf" proposa!s that simply await a win-
dow of opportuniry, and prepackaged proposals that polirical ideologues 5, See also rhe very valuabie discussion on generic policy inscrurnenrs in W~imer and Vining,
are perennially advocating. Poli'] AnaljJiJ, chap. 9.