AUTOCORRELATION

Presented by:
Durria Anjum
Lalarukh
Afira Iqbal
Nida Habib
(Scholars of M.Phill
Management Sciences)


Qurtuba University of Science and
Information Technology


OUTLINE
What is autocorrelation
Introduction
Structure
Consequences of autocorrelation
Detection of autocorrelation
Plot the residuals
Durbin Watson Test
The Breusch-Godfrey Lagrange Multiplier
Test
Remedies of autocorrelation
GLS Estimator
FGLS Estimator
WHAT IS AUTOCORRELATION
Autocorrelation occurs when the errors are
correlated. In this case, we can think of the
disturbances for different observations as
being drawn from different distributions
that are not explanatory distributions.



WHAT IS AUTOCORRELATION
Example
Suppose that we have the following equation
that describes the statistical relationship
between annual consumption expenditures and
annual disposable income for Pakistan for the
period 2001 to 2010. Thus, we have 10
multivariate observations on consumption
expenditures and income, one for each year.

Y
t
= o + |X
t
+
t
for t = 1, , 10

WHAT IS AUTOCORRELATION
Because of the time-series nature of the data
we would expect the disturbances for different
years to be correlated with one another. For
example, we might expect the disturbance in
year t to be correlated with the disturbance in
year t-1. In particular, we might expect a
positive correlation between the disturbances
in year t and year t-1.
WHAT IS AUTOCORRELATION
For example, if the disturbance in year t-1
(e.g., 2009) is positive, it is quite likely that the
disturbance in year t (2010) will be positive, or
if the disturbance in year t-1 (e.g., 2009) is
negative, it is quite likely that the disturbance
in year t (2010) will be negative. This is
because time-series data tends to follow trends.
The assumption of autocorrelation can be
expressed as follows
Cov(
t ,

s
) = E(
t

s
) = 0



WHAT IS AUTOCORRELATION
Thus, autocorrelation occurs whenever
the disturbance for period t is correlated
with the disturbance for period s. In the
above example, autocorrelation exists
because the disturbance in year t is
correlated with the disturbance in year t-
1.


WHAT IS AUTOCORRELATION
WHAT IS AUTOCORRELATION
Structure Of Autocorrelation
First-Order Autocorrelation
The model of autocorrelation that is
assumed most often is called the first-
order autoregressive process. This is most
often called AR(1). The AR(1) model of
autocorrelation assumes that the disturbance
in period t (current period) is related to the
disturbance in period t-1 (previous period).
WHAT IS AUTOCORRELATION
For the consumption function example, the
general linear regression model that assumes
an AR(1) process is given by
Y
t
= o + |X
t
+
t
for t = 1, , 10

t
=
t-1
+ c
t
where -1 < < 1


WHAT IS AUTOCORRELATION
The second equation tells us that the
disturbance in period t (current period)
depends upon the disturbance in period t-1
(previous period) plus some additional
amount, which is an error. In our example,
this assumes that the disturbance for the
current year depends upon the disturbance for
the previous year plus some additional amount
or error.
WHAT IS AUTOCORRELATION
The parameter is called the first-order
autocorrelation coefficient. Note that it is
assumed that can take any value
between negative one and positive one.
Thus, can be interpreted as the
correlation coefficient between
t
and
t-1
.
WHAT IS AUTOCORRELATION
If > 0, then the disturbances in period t are
positively correlated with the disturbances in
period t-1. In this case there is positive
autocorrelation. This means that when
disturbances in period t-1 are positive
disturbances, then disturbances in period t
tend to be positive. When disturbances in
period t-1 are negative disturbances, then
disturbances in period t tend to be negative.
Time-series data sets in economics are usually
characterized by positive autocorrelation.
WHAT IS AUTOCORRELATION
If < 0, then the disturbances in period t
are negatively correlated with the
disturbances in period t-1. In this case there
is negative autocorrelation. This means that
when disturbances in period t-1 are positive
disturbances, then disturbances in period t
tend to be negative. When disturbances in
period t-1 are negative disturbances, then
disturbances in period t tend to be positive.


WHAT IS AUTOCORRELATION
Second-Order Autocorrelation
An alternative model of autocorrelation is called
the second-order autoregressive process or
AR(2). The AR(2) model of autocorrelation
assumes that the disturbance in period t is
related to both the disturbance in period t-1
and the disturbance in period t-2. The general
linear regression model that assumes an AR(2)
process is given by
Y
t
= o + |X
t
+
t
for t = 1, , 10

t
=
1

t-1
+
2

t-2
+ c
t


WHAT IS AUTOCORRELATION
The second equation tells us that the
disturbance in period t depends upon the
disturbance in period t-1, the disturbance
in period t-2, and some additional
amount, which is an error. Once again, it
is assumed that these errors are
explanatory and identically distributed
with mean zero and constant variance.



WHAT IS AUTOCORRELATION
th-Order Autocorrelation
The general linear regression model that
assumes a th-order autoregressive process
or AR(), where can assume any positive
value is given by
Y
t
= o + |X
t
+
t
for t = 1, , 10

t
=
1

t-1
+
2

t-2
+ +

t-
+ c
t


WHAT IS AUTOCORRELATION
For example, if we have quarterly data on
consumption expenditures and disposable
income, we might argue that a fourth-order
autoregressive process is the appropriate
model of autocorrelation. However, once
again, the most often used model of
autocorrelation is the first-order autoregressive
process.



WHAT IS AUTOCORRELATION
CONSEQUENCES OF
AUTOCORRELATION
1. The OLS estimator is still unbiased.
2. The OLS estimator is inefficient; that is, it
is not BLUE.
3. The estimated variances and covariances
of the OLS estimates are biased and
inconsistent.

CONSEQUENCES OF
AUTOCORRELATION
If there is positive autocorrelation, and if
the value of a right-hand side variable
grows over time, then the estimate of the
standard error of the coefficient estimate
of this variable will be too low and hence
the t-statistic too high.
4. Hypothesis tests are not valid.


DETECTION OF AUTOCORRELATION
There are several ways to use the sample
data to detect the existence of
autocorrelation.
Plot the Residuals
The error for the t th observation,
t
, is
unknown and unobservable. However, we
can use the residual for the t th observation,

t
.
as an estimate of the error. One way to
detect autocorrelation is to estimate the
equation using OLS, and then plot the
residuals against time.

DETECTION OF AUTOCORRELATION
In our example, the residual would be
measured on the vertical axis. The years
2001 to 2010 would be measured on the
horizontal axis. We can then examine the
residual plot to determine if the residuals
appear to exhibit a pattern of correlation.
Most statistical packages have a command
that does this residual plot for us. It must
be emphasized that this is not a formal test
of autocorrelation. It would only suggest
whether autocorrelation may exist.
DETECTION OF AUTOCORRELATION
Note: We should not substitute a residual
plot for a formal test.
The Durbin-Watson d Test
The most often used test for first-order. It is
important to note that this test can only be
used to test for first-order autocorrelation, it
cannot be used to test for higher-order
autocorrelation. Also, this test cannot be
used if the lagged value of the dependent
variable is included as a right-hand side
variable.

DETECTION OF AUTOCORRELATION
Example
Suppose that the regression model is given
by
Y
t
= |
1
+ |
2
X
t2
+ |
3
X
t3
+
t


t
=
t-1
+ c
t
where -1 < < 1
Where Y
t
is annual consumption expenditures
in year t, X
t2
is annual disposable income in
year t, and X
t3
is the interest rate for year t.





DETECTION OF AUTOCORRELATION
We want to test for first-order positive
autocorrelation. Economists usually test for
positive autocorrelation because negative serial
correlation is highly unusual when using
economic data. The null and alternative
hypotheses are
H
0
: = 0
H
1
> 0
Note that this is a one-sided or one-tailed test.


DETECTION OF AUTOCORRELATION
To do the test, proceed as follows.
Step #1: Regress Y
t
against a constant, X
t2

and X
t3
using the OLS estimator.
Step #2: Use the OLS residuals from this
regression to calculate the following test
statistic:
d =
t=2
(
t
.

t-1
.
)
2
/
t=1
(
t
.
)
2


DETECTION OF AUTOCORRELATION
Note the following:
1. It can be shown that the test-statistic d can
take any value between 0 and 4.
2. It can be shown if d = 0, then there is
extreme positive autocorrelation.
3. It can be shown if d = 4, then there is
extreme negative autocorrelation.

DETECTION OF AUTOCORRELATION
5. It can be shown if d = 2, then there is no
autocorrelation.
Note: A rule of thumb that is sometimes
used is to conclude that there is no first-
order autocorrelation if the d statistic is
between 1.5 and 2.5. A d statistic below
1.5 indicates positive first-order
autocorrelation. A d statistic of greater
than 2.5 indicates negative first-order
autocorrelation. However, strictly
speaking, this is not correct.
DETECTION OF AUTOCORRELATION
Step #4: Compare the value of the test
statistic to the critical values using the
following decision rule. If d < d
L
then reject
the null and conclude there is first-order
autocorrelation.If d > d
u
then do accept
the null and conclude there is no first-order
autocorrelation. If d
L
s d s d
U
the test is
inconclusive.


DETECTION OF AUTOCORRELATION
DETECTION OF AUTOCORRELATION
The Breusch-Godfrey Lagrange Multiplier
Test
The Breusch-Godfrey test is a general test of
autocorrelation. It can be used to test for first-
order autocorrelation or higher-order
autocorrelation. This test is a specific type of
Lagrange multiplier test.



DETECTION OF AUTOCORRELATION
Example
Suppose that the regression model is given by
Y
t
= |
1
+ |
2
X
t2
+ |
3
X
t3
+
t


t
=
1

t-1
+
2

t-2
+ c
t
where -1 < < 1

Where Y
t
is annual consumption expenditures
in year t, X
t2
is annual disposable income in
year t, and X
t3
is the interest rate for year t.
We want to test for second-order
autocorrelation.
DETECTION OF AUTOCORRELATION
Economists usually test for positive
autocorrelation because negative serial
correlation is highly unusual when using
economic data. The null and alternative
hypotheses are
H
0
:
1
=
2
= 0
H
1
At least one is not zero
The logic of the test is as follows. Substituting
the expression for
t
into the regression
equation yields the following
Y
t
= |
1
+ |
2
X
t2
+ |
3
X
t3
+
1

t-1
+
2

t-2
+ c
t




DETECTION OF AUTOCORRELATION
To test the null-hypotheses of no
autocorrelation, we can use a Lagrange
multiplier test to whether the variables
t-1
and

t-2
belong in the equation. To do the test,
proceed as follows.
Step #1: Regress Y
t
against a constant, X
t2
and
X
t3
using the OLS estimator and obtain the
residuals
t
.
.
DETECTION OF AUTOCORRELATION
Step #2: Regress
t
.
against a constant, X
t2
,
X
t3
,
t-1
.
and
t-2
.
using the OLS estimator.
Note that for this regression we will have n-2
observations, because two observations must
be used to calculate the residual variables
t-1
.

and
t-2
.
. Thus, in our example we would run
this regression using the observations for the
period 2003 to 2010. We lose the
observations for the years 2001 and 2002.


DETECTION OF AUTOCORRELATION
Thus, we have 8 observations.
Step #3: Find the unadjusted R
2
statistic and
the number of observations, n 2, for the
auxiliary regression.
Step #4: Calculate the LM test statistic as
follows: LM = (n 2)R
2
.
Step #5: Choose the level of significance of
the test and find the critical value of LM. The
LM statistic has a chi-square distribution with
two degrees of freedom, _
2
(2).
DETECTION OF AUTOCORRELATION
Step #6: If the value of the test statistic,
LM, exceeds the critical value, then reject the
null and conclude that there is autocorrelation.
If not, accept the null and conclude that
there is no autocorrelation.



REMEDIES FOR AUTOCORRELATION
If the true model of the data generation
process is characterized by autocorrelation,
then the best linear unbiased estimator
(BLUE) is the generalized least squares
(GLS) estimator.



REMEDIES FOR AUTOCORRELATION
GLS Estimator
Deriving the GLS Estimator for a General
Linear Regression Model with First-
Order Autocorrelation
Suppose that we have the following general
linear regression model. For example, this
may be the consumption expenditures model.
Y
t
= o + |X
t
+
t
for t = 1, , n

t
=
t-1
+ c
t



REMEDIES FOR AUTOCORRELATION
Recall that the error term c
t
satisfies the
assumptions of the classical linear regression
model. This statistical model describes what
we believe is the true underlying process that
is generating the data. To derive the GLS
estimator, we proceed as follows.
1. Derive a transformed model that satisfies
all of the assumptions of the classical linear
regression model.
REMEDIES FOR AUTOCORRELATION
2. Apply the OLS estimator to the
transformed model.

The GLS estimator is the OLS estimator
applied to the transformed model. To
derive the transformed model, proceed as
follows. Substitute the expression for
t

into the regression equation. Doing so
yields
(*) Y
t
= o + |X
t
+
t-1
+ c
t

REMEDIES FOR AUTOCORRELATION
If we can eliminate the term
t-1
from this
equation, we would be left with the error term
c
t
that satisfies all of the assumptions of the
classical linear regression model, including the
assumption of no autocorrelation. To
eliminate
t-1
from the equation, we proceed
as follows. The original regression equation Y
t

= o + |X
t
+
t
must be satisfied for every
single observation.

REMEDIES FOR AUTOCORRELATION
Therefore, this equation must be satisfied in
period t 1 as well as in period t. Therefore,
we can write,
Y
t-1
= o + |X
t-1
+
t-1

This is called lagging the equation by one time
period. Solving this equation for
t-1
yields

t-1
= Y
t-1
- o - |X
t-1


REMEDIES FOR AUTOCORRELATION
Now, multiply each side of this equation
by the parameter . Doing so yields

t-1
= Y
t-1
- o - |X
t-1

Substituting this expression for
t-1
into
equation (*) yields
Y
t
= o + |X
t
+ Y
t-1
- o - |X
t-1
+ c
t


REMEDIES FOR AUTOCORRELATION
This can be written equivalently as
Y
t
- Y
t-1
= (1 - )o + |(X
t
- X
t-1
) + c
t

This can be written equivalently as
Y
t
*
= o
*
+ |X
t
*
+ c
t

Where Y
t
*
= Y
t
- Y
t-1
; o
*
= (1 - )o
; X
t
*
= X
t
- X
t-1
.
REMEDIES FOR AUTOCORRELATION
This is the transformed model. Note the
following:
1. The slope coefficient of the transformed
model, |, is the same as the slope
coefficient of the original model.
2. The constant term in the original model
is given by o = o
*
/(1 - ).




REMEDIES FOR AUTOCORRELATION
3. The error term in the transformed model,
c
t
, satisfies all of the assumptions of the
error term in the classical linear regression
model.
Thus, if we run a regression of the
transformed variable Y
t
*
on a constant and the
transformed variable X
t
*
using the OLS
estimator, we can get a direct estimate of |
and solve for the estimate of o. These
estimates are not GLS estimates and therefore
are not BLUE.
REMEDIES FOR AUTOCORRELATION
The problem is that when we create the
transformed variables Y
t
*
and X
t
*
we lose
one observation because Y
t-1
and X
t-1
are
lagged one period. Therefore, we have n 1
observations to estimate the transformed
model.

REMEDIES FOR AUTOCORRELATION
In our example, we would lose the
observation for the first year, which is 2001. It
can be shown that to preserve the first
observation (n = 1), we can use the following.

Y
1
*
= (1 -
2
)
1/2
; X
1
*
= X
1
/ (1 -
2
)
1/2



REMEDIES FOR AUTOCORRELATION
The GLS estimator involves the following
steps.

1. Create the transformed variables Y
t
*
and
X
t
*
.
2. Regress the transformed variable Y
t
*
on a
constant and the transformed variable X
t
*

using the OLS estimator and all n
observations.
The resulting estimates are GLS estimates,
which are BLUE.


REMEDIES FOR AUTOCORRELATION
Problems with Using the GLS Estimator
The major problem with the GLS estimator is
that to use it you must know the true
autocorrelation coefficient . If you dont the
value of , then you cant create the
transformed variables Y
t
*
and X
t
*
. However,
the true value of is almost always unknown
and unobservable. Thus, the GLS is not a
feasible estimator.


REMEDIES FOR AUTOCORRELATION
Feasible Generalized Least Squares
(FGLS) Estimator
The GLS estimator requires that we know the
value of . To make the GLS estimator
feasible, we can use the sample data to obtain
an estimate of . When we do this, we have a
different estimator. This estimator is called
the Feasible Generalized Least Squares
Estimator, or FGLS estimator.
REMEDIES FOR AUTOCORRELATION
The two most often used FGLS estimators are:
1. Cochrane-Orcutt estimator
2. Hildreth-Lu estimator
Example
Suppose that we have the following general
linear regression model. For example, this
may be the consumption expenditures model.
Y
t
= o + |X
t
+
t
for t = 1, , n

t
=
t-1
+ c
t




REMEDIES FOR AUTOCORRELATION
Recall that the error term c
t
satisfies the
assumptions of the classical linear
regression model. This statistical model
describes what we believe is the true
underlying process that is generating
the data.

REMEDIES FOR AUTOCORRELATION
Cochrane-Orcutt Estimator
To obtain FGLS estimates of o and | using the
Cochrane-Orcutt estimator, proceed as follows.
Step #1: Regress Y
t
on a constant and X
t

using the OLS estimator.
Step #2: Calculate the residuals from this
regression,
t
.
.

REMEDIES FOR AUTOCORRELATION
Step #3: Regress
t
.
on
t-1
.
using the OLS
estimator. Do not include a constant term in
the regression.This yields an estimate of ,
denoted
.
.
Step #4: Use the estimate of to create the
transformed variables: Y
t
*
= Y
t
- Y
t-1
, X
t
*
=
X
t
- X
t-1
.

REMEDIES FOR AUTOCORRELATION
Step #5: Regress the transformed variable Y
t
*

on a constant and the transformed variable X
t
*

using the the OLS estimator.
Step #6: Use the estimate of o and | from
step #5 to get calculate a new set of
residuals,

t
.
.
Step #7: Repeat step #2 through step #6.

REMEDIES FOR AUTOCORRELATION
Step #8: Continue iterating step #2 through
step #5 until the estimate of from two
successive iterations differs by no more than
some small predetermined value, such as
0.001.
Step #9: Use the final estimate of to get the
final estimates of o and |.


REMEDIES FOR AUTOCORRELATION
Hildreth-Lu Estimator
To obtain FGLS estimates of o and | using
the Hildreth-Lu estimator, proceed as follows.
Step #1: Choose a value of of between 1
and 1.
Step #2: Use the this value of to create the
transformed variables: Y
t
*
= Y
t
- Y
t-1
, X
t
*
= X
t

- X
t-1
.


REMEDIES FOR AUTOCORRELATION
Step #3: Regress the transformed variable Y
t
*

on a constant and the transformed variable X
t
*

using the the OLS estimator.
Step #4: Calculate the residual sum of
squares
for this regression.
Step #5: Choose a different value of of
between 1 and 1.
Step #6: Repeat step #2 through step #4.
REMEDIES FOR AUTOCORRELATION
Step #7: Repeat Step #5 and step #6. By
letting vary between 1 and 1in a systematic
fashion, you get a set of values for the
residual sum of squares, one for each
assumed value of .
Step #8: Choose the value of with the
smallest residual sum of squares.
Step #9: Use this estimate of to get the
final estimates of o and |.


REMEDIES FOR AUTOCORRELATION
Comparison of the Two Estimators
If there is more than one local minimum for
the residual sum of squares function, the
Cochrane-Orcutt estimator may not find the
global minimum. The Hildreth-Lu estimator
will find the global minimum. Most statistical
packages have both estimators. Some
econometricians suggest that you estimate
the model using both estimators to make sure
that the Cochrane-Orcutt estimator doesnt
miss the global minimum.


REMEDIES FOR AUTOCORRELATION
Properties of the FGLS Estimator
If the model of autocorrelation that you
assume is a reasonable approximation of the
true autocorrelation, then the FGLS estimator
will yield more precise estimates than the OLS
estimator. The estimates of the variances and
covariances of the parameter estimates will
also be unbiased and consistent.
REMEDIES FOR AUTOCORRELATION
However, if the model of autocorrelation that
you assume is not a reasonable approximation
of the true autocorrelation, then the FGLS
estimator will yield worse estimates than the
OLS estimator.


REMEDIES FOR AUTOCORRELATION
Generalizing the Model
The above examples assume that there is one
explanatory variable and first-order
autocorrelation. The model and FGLS
estimators can be easily generalized to the
case of k explanatory variables and higher-
order autocorrelation.

Thanks