Chapter 11

Using ACF and PACF to select MA(q) or AR(p)

models
The principle way to determine which AR or MA model is appropriate is to look at the ACF and PACF of
the time series. The table 11.1 gives the theoretical behaviour of these functions for different MA and AR
models. An informal way to pick a model is to compute the ACF and PACF for a time series and match it to
the behaviour in the table 11.1. This rule is however difcult to use when the time series is explained by an
ARMA model (combined effect of AR and MA).
MODEL ACF PACF
AR(1) Exponential decay: on +ve side if
1
> 0
and alternating in sign, starting on ve
side, if
1
<0.
Spike at lag 1, then 0; +ve spike if
1
> 0
and ve spike if
1
<0.
AR(p) Exponential decay or damped sine wave.
The exact pattern depends on the signs
and sizes of
1
, . . . ,
p
.
Spikes at lags 1 to p, then zero.
MA(1) Spike at lag 1, then 0; +ve spike if
1
< 0
and ve spike if
1
>0.
Exponential decay: on +ve side if
1
< 0
and alternating in sign, starting on +ve
side, if
1
<0.
MA(q) Spikes at lags 1 to q, then zero. Exponential decay or damped sine wave.
The exact pattern depends on the signs
and sizes of
1
, . . . ,
q
.
Table 11.1: Shapes of ACF and PACF to identify AR or MA models suitable to t time series.
11.1 ACF and PACF
1.1 Denition (ACF) At lag k, the ACF is computed by:
ACF(k) =
E
__
y
t
E[y
t
]
__
y
t k
E[y
t k
]
__
_
Var [y
t
]Var [y
t k
]
In time series, we may want to measure the relationship between Y
t
and Y
t k
when the effects of other
time lags 1, 2, . . . , k 1 have been removed. The autocorrelation does not measure this. However, Partial
autocorrelation is a way to measure this effect.
40
1.2 Denition (PACF) The partial autocorrelation of a time series at lag k is denoted
k
and is found as
follows:
(1) Fit a linear regression of y
t
to the rst k lags (i.e. t an AR(k) model to the time series):
y
t
=
0
+
1
y
t 1
+
2
y
t 2
+ +
k
y
t k
+
t
(2) Then
k
=

k
, the tted value of
k
from the regression (Least Squares).
The set of partial autocorrelations at different lags is called the partial autocorrelation function (PACF) and
is plotted like the ACF.
11.2 Exercises: ACF and PACF for AR(1) and MA(1)
(1) Assuming AR(1) model with
0
=0, show that the PACF coefcients are zeros when k >1.
Ans. By denition, the model is (ignoring the constant term
0
) :
y
t
=
1
y
t 1
+
t
Computing the PACF at order 2 for instance, implies to t a AR(2) model to our AR(1). This is easily
done:
y
t
=
1
y
t 1
+0 y
t 2
+
t
therefore the PACF coefcient at lag 2, is 0. The same reasoning can be used for any k >1. At lag k =1,
the PACF coefcient is
1
. This explains the shape of the PACF you have for a simulated AR(1) model
using R.
(2) Lets assume a MA(1) model with
0
=0
what is E[y
t
]?
Ans.
E[y
t
] =E[
1

t 1
+
t
] By def. of our MA(1)
=
1
E[
t 1
] +E[
t
] Expectation is a linear operator
=
1
0+0 Since
t
N (0,
2
) t (i.e. expectation of the errors is 0)
=0
What is the variance of y
t
?
Ans.
Var [y
t
] =E[(y
t
E[y
t
])
2
] By def. of variance
=E[(y
t
)
2
] since E[y
t
] =0
=E[(
1

t 1
+
t
)
2
] By def. of our MA(1)
=E[
2
1

2
t 1
+
2
t
+2
1

t 1

t
]
=
2
1
E[
2
t 1
] +E[
2
t
] +2
1
E[
t 1

t
]
=
2
1

2
+
2
+2
1
0 Using the hypothesis on the errors
Remember that all errors followa Normal distribution with mean 0 (E[
t
] =0, t ), and variance

2
. In addition, the errors are independent from each other i.e.:
E[
t
1

t
2
] =0 t
1
=t
2
41 Rozenn Dahyot 2012
What is the covariance of y
t
and y
t k
?
Ans.
Cov[y
t
, y
t k
] =E[(y
t
E[y
t
])(y
t k
E[y
t k
])] By def. of covariance
=E[(y
t
)(y
t k
)] Because E[y
t
] =0 t
=E[(
1

t 1
+
t
)(
1

t 1k
+
t k
)] Because of our MA(1) model
=E[
2
1

t 1k

t 1
+
1

t 1k

t
+
1

t 1k

t
+
t 1

t k
)]
=
2
1
E[
t 1k

t 1
]
. .
=0,k1
+
1
E[
t k

t 1
]
. .
0,k>1;
2
f or k=1
+
1
E[
t

t 1k
]
. .
0,k
+E[
t

t k
)]
. .
0,k>0
so Cov[y
t
, y
t 1
] =
2
and Cov[y
t
, y
t k
] =0, k >1.
What is the correlation of y
t
and y
t k
?
Ans. The correlation is the covariance divided by the variances:
Cor r [y
t
, y
t k
] =
Cov[y
t
, y
t k
]
_
Var [y
t
]Var [y
t k
]
=
_

1

2
1
+1
if k =1
0 otherwise t >1
(3) Conclude about the form of the ACF function for a MA(1) models?
Ans. The ACF plots the lags k on the x-axis, and the y-axis reports the correlation Cor r [y
t
, y
t k
]. Hence,
the ACF is equal to 1 at lag k =0, the ACF is equal to
1
at lag k =1, then the ACF is zeros for all other
lags k >1.
11.3 Least Squares algorithmfor MA models ?
Consider an MA(1) (with
0
=0 for simplication)
y
t
=
1

t 1
+
t
we need to write this with lagged series of y
t
(for which we have the observations y
1
, , y
n
). The model can
be rewritten:
y
t
=
1
y
t 1

2
1

t 2
+
t
=
1
y
t 1

2
1
y
t 2
+
3
1

t 3
+
t
.
.
.
y
t
=
1
y
t 1

2
1
y
t 2
+ +(1)
t

t 1
1
y
1
+
t
1

0
+
t
Assuming
0
=0, y
t
is a weighted average of all the past observations, and the expression is not linear w.r.t.
the parameter to estimate
1
(powers of
1
appear in the equation). Hence the Least square algorithmused
for estimation with AR models cannot be used.
42 Rozenn Dahyot 2012