Difference in Difference

Models
Bill Evans
Spring 2008

1
Difference in difference models
• Maybe the most popular identification
strategy in applied work today
• Attempts to mimic random assignment
with treatment and “comparison” sample
• Application of two-way fixed effects model

2
 Problem set up
• Cross-sectional and time series data
• One group is ‘treated’ with intervention
• Have pre-post data for group receiving
intervention
• Can examine time-series changes but,
unsure how much of the change is due to
secular changes

3
Y

True effect = Yt2-Yt1

Estimated effect = Yb-Ya

Yt1

Ya

Yb

Yt2

t1 ti t2
time
4
• Intervention occurs at time period t1
• True effect of law
– Ya – Yb
• Only have data at t1 and t2
– If using time series, estimate Yt1 – Yt2
• Solution?

5
Difference in difference models
• Basic two-way fixed effects model
– Cross section and time fixed effects
• Use time series of untreated group to
establish what would have occurred in the
absence of the intervention
• Key concept: can control for the fact that
the intervention is more likely in some
types of states
6
 Three different presentations
• Tabular
• Graphical
• Regression equation

7
 Difference in Difference
Before After
Change Change Difference
Group 1 Yt1 Yt2 ΔYt
(Treat) = Yt2-Yt1
Group 2 Yc1 Yc2 ΔYc
(Control) =Yc2-Yc1
Difference ΔΔY
ΔYt – ΔYc
8
Y
Treatment effect=
(Yt2-Yt1) – (Yc2-Yc1)

Yc1

Yt1

Yc2

Yt2
control

treatment

t1 t2
time
9
 Key Assumption
• Control group identifies the time path of
outcomes that would have happened in
the absence of the treatment
• In this example, Y falls by Yc2-Yc1 even
without the intervention
• Note that underlying ‘levels’ of outcomes
are not important (return to this in the
regression equation)

10
 Y

Yc1 Treatment effect=

(Yt2-Yt1) – (Yc2-Yc1)

Yc2
Yt1

control

Yt2 Treatment
Effect

treatment

t1 t2
time
11
• In contrast, what is key is that the time
trends in the absence of the intervention
are the same in both groups
• If the intervention occurs in an area with a
different trend, will under/over state the
treatment effect
• In this example, suppose intervention
occurs in area with faster falling Y

12
 Y

Estimated treatment
Yc1

Yt1
Yc2

True treatment effect control

Yt2 True
Treatment
treatment Effect

t1 t2
time
13
 Basic Econometric Model
• Data varies by
– state (i)
– time (t)
– Outcome is Yit
• Only two periods
• Intervention will occur in a group of
observations (e.g. states, firms, etc.)

14
• Three key variables
– Tit =1 if obs i belongs in the state that will
eventually be treated
– Ait =1 in the periods when treatment occurs
– TitAit -- interaction term, treatment states after
the intervention
• Yit = β0 + β1Tit + β2Ait + β3TitAit + εit

15
 Yit = β0 + β1Tit + β2Ait + β3TitAit + εit
Before After
Change Change Difference
Group 1 β0 + β1 β0+ β1+ β2+ β3 ΔYt
(Treat) = β2 + β3
Group 2 β0 β0 + β2 ΔYc
(Control) = β2
Difference ΔΔY = β3

16
 More general model
• Data varies by
– state (i)
– time (t)
– Outcome is Yit
• Many periods
• Intervention will occur in a group of states
but at a variety of times

17
• ui is a state effect
• vt is a complete set of year (time) effects
• Analysis of covariance model

• Yit = β0 + β3 TitAit + ui + λt + εit

18
 What is nice about the model
• Suppose interventions are not random but
systematic
– Occur in states with higher or lower average Y
– Occur in time periods with different Y’s
• This is captured by the inclusion of the
state/time effects – allows covariance
between
– ui and TitAit
– λt and TitAit

19
• Group effects
– Capture differences across groups that are
constant over time
• Year effects
– Capture differences over time that are
common to all groups

20
 Meyer et al.
• Workers’ compensation
– State run insurance program
– Compensate workers for medical expenses
and lost work due to on the job accident
• Premiums
– Paid by firms
– Function of previous claims and wages paid
• Benefits -- % of income w/ cap
21
• Typical benefits schedule
– Min( pY,C)
– P=percent replacement
– Y = earnings
– C = cap

– e.g., 65% of earnings up to $400/month

22
• Concern:
– Moral hazard. Benefits will discourage return to work
• Empirical question: duration/benefits gradient
• Previous estimates
– Regress duration (y) on replaced wages (x)
• Problem:
– given progressive nature of benefits, replaced wages
reveal a lot about the workers
– Replacement rates higher in higher wage states

23
• Yi = Xiβ + αRi + εi
• Y (duration)
• R (replacement rate)
• Expect α > 0
• Expect Cov(Ri, εi)
– Higher wage workers have lower R and
higher duration (understate)
– Higher wage states have longer duration and
longer R (overstate)
24
 Solution
• Quasi experiment in KY and MI
• Increased the earnings cap
– Increased benefit for high-wage workers
• (Treatment)
– Did nothing to those already below original
cap (comparison)
• Compare change in duration of spell
before and after change for these two
groups
25
26
27
 Model
• Yit = duration of spell on WC
• Ait = period after benefits hike
• Hit = high earnings group (Income>E3)

• Yit = β0 + β1Hit + β2Ait + β3AitHit + β4Xit’ + εit

• Diff-in-diff estimate is β3

28
29
 Questions to ask?
• What parameter is identified by the quasi-
experiment? Is this an economically
meaningful parameter?
• What assumptions must be true in order
for the model to provide and unbiased
estimate of β3?
• Do the authors provide any evidence
supporting these assumptions?

30