ARIMA and GARCH

Introduction to ARMA and GARCH processes
Fulvio Corsi
SNS Pisa
3 March 2010
Fulvio Corsi Introduction to ARMA

() and GARCH processes SNS Pisa 3 March 2010 1 / 24
Stationarity
Strict stationarity:
d
(X1 , X2 , ..., Xn ) = (X1+k , X2+k , ..., Xn+k ) for any integer n > 1, k
Weak/second-order/covariance stationarity:
E[xt ] = µ
E[Xt − µ]2 = σ 2 < +∞ (i.e. constant and indipendent of t)
E[(Xt − µ)(Xt+k − µ)] = γ(|k|) (i.e. indipendent of t for each k)
Interpretation:
mean and variance are constant
mean reversion
shocks are transient
covariance between Xt and Xt−k tends to 0 as k → ∞

Stationarity
d
E[xt ] = µ
Interpretation:
mean reversion

Stationarity
d
E[xt ] = µ
Interpretation:
mean reversion

Withe noise
weak (uncorrelated)
E(ǫt ) = 0 ∀t
V(ǫt ) = σ 2 ∀t
γ(|t−s|)
ρ(ǫt , ǫs ) = 0 ∀s 6= t where ρ≡ γ(0)
strong (independence)
ǫt ∼ I.I.D.(0, σ 2 )
Gaussian (weak=strong)
ǫt ∼ N.I.D.(0, σ 2 )

Lag operator
the Lag operator is defined as:
LXt ≡ Xt−1
is a linear operator:
L(βXt ) = β · LXt = βXt−1
L(Xt + Yt ) = LXt + LYt = Xt−1 + Yt−1
and admits power exponent, for instance:

L2 Xt = L(LXt ) = LXt−1 = Xt−2
k
L Xt = Xt−k
L−1 Xt = Xt+1
Some examples:
∆Xt = Xt − Xt−1 = Xt − LXt = (1 − L)Xt
yt = (θ1 + θ2 L)LXt = (θ1 L + θ2 L2 )Xt = θ1 Xt−1 + θ2 Xt−2
Expression like
(θ0 + θ1 L + θ2 L2 + ... + θn Ln )
with possibly n = ∞, are called lag polynomial and are indicated as θ(L)

Lag operator
LXt ≡ Xt−1

k
L Xt = Xt−k
L−1 Xt = Xt+1
Some examples:
∆Xt = Xt − Xt−1 = Xt − LXt = (1 − L)Xt
Expression like
(θ0 + θ1 L + θ2 L2 + ... + θn Ln )

Lag operator
LXt ≡ Xt−1

k
L Xt = Xt−k
L−1 Xt = Xt+1
Some examples:
∆Xt = Xt − Xt−1 = Xt − LXt = (1 − L)Xt
Expression like
(θ0 + θ1 L + θ2 L2 + ... + θn Ln )

Lag operator
LXt ≡ Xt−1

k
L Xt = Xt−k
L−1 Xt = Xt+1
Some examples:
∆Xt = Xt − Xt−1 = Xt − LXt = (1 − L)Xt
Expression like
(θ0 + θ1 L + θ2 L2 + ... + θn Ln )

Moving Average (MA) process
The
P ∞simplest way to construct a stationary process is to use a lag polynomial θ(L) with
2
j=0 θj < ∞ to construct a sort of “weighted moving average” of withe noises ǫt , i.e.
MA(q)
Yt = θ(L)ǫt = ǫt + θ1 ǫt−1 + θ2 ǫt−2 + ... + +θq ǫt−q
Example, MA(1)
Yt = ǫt + θǫt−1 = (1 + θL)ǫt
being E[Yt ] = 0
γ(0) = E[Yt Yt ] = E[(ǫt + θǫt−1 )(ǫt + θǫt−1 )] = σ 2 (1 + θ2 );
γ(1) = E[Yt Yt−1 ] = E[(ǫt + θǫt−1 )(ǫt−1 + θǫt−2 )] = σ 2 θ;
γ(k) = E[Yt Yt−k ] = E[(ǫt + θǫt−1 )(ǫt−k + θǫt−k−1 )] = 0 ∀k > 1
and,
γ(1) θ
ρ(1) = =
γ(0) 1 + θ2
γ(k)
ρ(k) = =0 ∀k > 1
γ(0)
hence, while a withe noise is “0-correlated”, MA(1) is 1-correlated

(i.e. it has only the first correlation ρ(1) different from zero)
Moving Average (MA) process
The
P ∞simplest way to construct a stationary process is to use a lag polynomial θ(L) with
2
j=0 θj < ∞ to construct a sort of “weighted moving average” of withe noises ǫt , i.e.
MA(q)
Yt = θ(L)ǫt = ǫt + θ1 ǫt−1 + θ2 ǫt−2 + ... + +θq ǫt−q
Example, MA(1)
Yt = ǫt + θǫt−1 = (1 + θL)ǫt
being E[Yt ] = 0
γ(0) = E[Yt Yt ] = E[(ǫt + θǫt−1 )(ǫt + θǫt−1 )] = σ 2 (1 + θ2 );
γ(1) = E[Yt Yt−1 ] = E[(ǫt + θǫt−1 )(ǫt−1 + θǫt−2 )] = σ 2 θ;
γ(k) = E[Yt Yt−k ] = E[(ǫt + θǫt−1 )(ǫt−k + θǫt−k−1 )] = 0 ∀k > 1
and,
γ(1) θ
ρ(1) = =
γ(0) 1 + θ2
γ(k)
ρ(k) = =0 ∀k > 1
γ(0)
hence, while a withe noise is “0-correlated”, MA(1) is 1-correlated

(i.e. it has only the first correlation ρ(1) different from zero)
Properties MA(q)
In general for a MA(q) process we have
γ(0) = σ 2 (1 + θ12 + θ22 + ... + θq2 )
q−k
X
γ(k) = σ2 θj θj+k ∀k ≤ q
j=0
= 0 ∀k > q
and
P q−k
j=0θj θj+k
ρ(k) = Pq 2
∀k ≤ q
1+ j=1 θj
= 0 ∀k > q
Hence, an MA(q) is q-correlated and it can also be shown that any stationary q-correlated
process can be represented as an MA(q).
Wold Theorem: any mean zero covariance stationary process can be represented in the
form, MA(∞) + deterministic component (the two being uncorrelated).
But, given a q-correlated process, is the MA(q) process unique? In general no, indeed it can
be shown that for a q-correlated process there are 2q possible MA(q) with same
autocovariance structure. However, there is only one MA(q) which is invertible.

Properties MA(q)
γ(0) = σ 2 (1 + θ12 + θ22 + ... + θq2 )
q−k
X
j=0
= 0 ∀k > q
and
P q−k
j=0θj θj+k
ρ(k) = Pq 2
∀k ≤ q
1+ j=1 θj
= 0 ∀k > q

Properties MA(q)
γ(0) = σ 2 (1 + θ12 + θ22 + ... + θq2 )
q−k
X
j=0
= 0 ∀k > q
and
P q−k
j=0θj θj+k
ρ(k) = Pq 2
∀k ≤ q
1+ j=1 θj
= 0 ∀k > q

Invertibility conditions for MA
first consider the MA(1) case:
Yt = (1 + θL)ǫt
given the result
∞
X
(1 + θL)−1 = (1 − θL + θ2 L2 − θ3 L3 + θ4 L4 + ...) = (−θL)i
i=0
inverting the θ(L) lag polynomial, we can write
(1 − θL + θ2 L2 − θ3 L3 + θ4 L4 + ...)Yt = ǫt
which can be considered an AR(∞) process.
If an MA process can be written as an AR(∞) of this type, such MA representation is said to

be invertible. For MA(1) process the invertibility condition is given by |θ| < 1.
For a general MA(q) process

Yt = (1 + θ1 L + θ2 L2 + ... + θq Lq )ǫt
the invertibility conditions are that the roots of the lag polynomial
1 + θ1 z + θ2 z2 + ... + θq zq = 0
lie outside the unit circle. Then the MA(q) can be written as an AR(∞) by inverting θ(L).
Invertibility also has important practical consequence in application. In fact, given that the ǫt
are not observable they have to be reconstructed from the observed Y’s through the
AR(∞) representation.
Invertibility conditions for MA
first consider the MA(1) case:
Yt = (1 + θL)ǫt
given the result
∞
X
(1 + θL)−1 = (1 − θL + θ2 L2 − θ3 L3 + θ4 L4 + ...) = (−θL)i
i=0
inverting the θ(L) lag polynomial, we can write
(1 − θL + θ2 L2 − θ3 L3 + θ4 L4 + ...)Yt = ǫt
which can be considered an AR(∞) process.
If an MA process can be written as an AR(∞) of this type, such MA representation is said to

be invertible. For MA(1) process the invertibility condition is given by |θ| < 1.
For a general MA(q) process

Yt = (1 + θ1 L + θ2 L2 + ... + θq Lq )ǫt
the invertibility conditions are that the roots of the lag polynomial
1 + θ1 z + θ2 z2 + ... + θq zq = 0
lie outside the unit circle. Then the MA(q) can be written as an AR(∞) by inverting θ(L).
Invertibility also has important practical consequence in application. In fact, given that the ǫt
are not observable they have to be reconstructed from the observed Y’s through the
AR(∞) representation.
Auto-Regressive Process (AR)
A general AR process is defined as
φ(L)Yt = ǫt
It is always invertible but not always stationary.
Example: AR(1)
(1 − φL)Yt = ǫt or Yt = φYt−1 + ǫt
by inverting the lag polynomial (1 − φL) the AR(1) can be written as
∞
X ∞
X
Yt = (1 − φL)−1 ǫt = (φL)i ǫt = φi ǫt−i = MA(∞)
i=0 i=0
hence the stationarity condition is that |φ| < 1.
From this representation we can apply the general formula of MA to compute γ(·) and ρ(·).
In particular,
ρ(k) = φ|k| ∀k
i.e. monotonic exponential decay for φ > 0 and exponentially damped oscillatory decay for
φ < 0.
In general an AR(p) process
Yt = φ1 Yt−1 + φ2 Yt−2 + ... + φp Yt−p + ǫt
is stationarity if all the roots of the characteristic equation of the lag polynomial
1 − φ1 z − φ2 z2 − ... − φp zp = 0
are outside the unit circle.
Auto-Regressive Process (AR)
A general AR process is defined as
φ(L)Yt = ǫt
It is always invertible but not always stationary.
Example: AR(1)
(1 − φL)Yt = ǫt or Yt = φYt−1 + ǫt
by inverting the lag polynomial (1 − φL) the AR(1) can be written as
∞
X ∞
X
Yt = (1 − φL)−1 ǫt = (φL)i ǫt = φi ǫt−i = MA(∞)
i=0 i=0
hence the stationarity condition is that |φ| < 1.
From this representation we can apply the general formula of MA to compute γ(·) and ρ(·).
In particular,
ρ(k) = φ|k| ∀k
i.e. monotonic exponential decay for φ > 0 and exponentially damped oscillatory decay for
φ < 0.
In general an AR(p) process
is stationarity if all the roots of the characteristic equation of the lag polynomial
1 − φ1 z − φ2 z2 − ... − φp zp = 0
are outside the unit circle.
State Space Representation of AR(p)
to gain more intuition on the AR stationarity conditions write an AR(p) in its state space form
      
Yt φ1 φ2 φ3 . . . φp−1 φp Yt−1 ǫt
 Yt−1   1 0 0 ... 0 0   Yt−2   0 
      
 .  =  . . . . .  . + . 
 ..   .. .. .. ... .. ..   ..   .. 
Yt−p+1 0 0 0 ... 1 0 Yt−p 0
Xt = F Xt−1 + vt
Hence, the expected value of Xt satisfy,
E[Xt ] = F Xt−1 and E[Xt+j ] = F j+1 Xt−1
is a linear map in Rp whose dynamic properties are given by the eigenvalues of the matrix F.
The eigenvalues of F are given by solving the characteristic equation

λp − φ1 λp−1 − φ2 λp−2 − ... − φp−1 λ − φp = 0.
Comparing this with the characteristic equation of the lag polynomial φ(L)
1 − φ1 z − φ2 z2 − ... − φp−1 zp−1 − φp zp = 0
we can see that the roots of the 2 equations are such that
z1 = λ−1
1 , z2 = λ−1
2 , ... , zp = λ−1
p

State Space Representation of AR(p)
to gain more intuition on the AR stationarity conditions write an AR(p) in its state space form
      
Yt φ1 φ2 φ3 . . . φp−1 φp Yt−1 ǫt
 Yt−1   1 0 0 ... 0 0   Yt−2   0 
      
 .  =  . . . . .  . + . 
 ..   .. .. .. ... .. ..   ..   .. 
Yt−p+1 0 0 0 ... 1 0 Yt−p 0
Xt = F Xt−1 + vt
Hence, the expected value of Xt satisfy,
E[Xt ] = F Xt−1 and E[Xt+j ] = F j+1 Xt−1
is a linear map in Rp whose dynamic properties are given by the eigenvalues of the matrix F.
The eigenvalues of F are given by solving the characteristic equation

λp − φ1 λp−1 − φ2 λp−2 − ... − φp−1 λ − φp = 0.
Comparing this with the characteristic equation of the lag polynomial φ(L)
1 − φ1 z − φ2 z2 − ... − φp−1 zp−1 − φp zp = 0
we can see that the roots of the 2 equations are such that
z1 = λ−1
1 , z2 = λ−1
2 , ... , zp = λ−1
p

ARMA(p,q)
An ARMA(p,q) process is defined as

φ(L)Yt = θ(L)ǫt
where φ(·) and θ(·) are pth and qth lag polynomials.
the process is stationary if all the roots of

φ(z) ≡ 1 − φ1 z − φ2 z2 − ... − φp−1 zp−1 − φp zp = 0
lie outside the unit circle and, hence, admits the MA(∞) representation:
Yt = φ(L)−1 θ(L)ǫt
the process is invertible if all the roots of

θ(z) ≡ 1 + θ1 z + θ2 z2 + ... + θq zq = 0
lie outside the unit circle and, hence, admits the AR(∞) representation:
ǫt = θ(L)−1 φ(L)Yt

Estimation
For AR(p)
OLS are consistent and under gaussianity asymptotically equivalent to MLE

→ asymptotically efficient
For example, the full Likelihood of an AR(1) can be written as

T
Y
L(θ) ≡ f (yT , yT−1 , ..., y1 ; θ) = fY1 (y1 ; θ) · fYt |Yt−1 (yt |yt−1 ; θ)
| {z } t=2
marginal first obs | {z }
conditional likelihood
under normality OLS=MLE
For a general ARMA(p,q)

Yt = φ1 Yt−1 + ... + φp Yt−p + ǫt + θ1 ǫt−1 + ... + θq ǫt−q
Yt−1 is correlated with ǫt−1 , ..., ǫt−q ⇒ OLS not consistent.
→ MLE with numerical optimization procedures.

Estimation
For AR(p)
OLS are consistent and under gaussianity asymptotically equivalent to MLE

→ asymptotically efficient
For example, the full Likelihood of an AR(1) can be written as

T
Y
L(θ) ≡ f (yT , yT−1 , ..., y1 ; θ) = fY1 (y1 ; θ) · fYt |Yt−1 (yt |yt−1 ; θ)
| {z } t=2
marginal first obs | {z }
conditional likelihood
under normality OLS=MLE
For a general ARMA(p,q)

Yt = φ1 Yt−1 + ... + φp Yt−p + ǫt + θ1 ǫt−1 + ... + θq ǫt−q
Yt−1 is correlated with ǫt−1 , ..., ǫt−q ⇒ OLS not consistent.
→ MLE with numerical optimization procedures.

Prediction
write the model in its AR(∞) representation:

η(L)(Yt − µ) = ǫt
then the optimal prediction of Yt+s is given by

η(L)−1 h i
E[Yt+s |Yt , Yt−1 , ...] = µ + η(L)(Yt − µ) with Lk = 0 for k < 0
Ls + +
which is known as Wiener-Kolmogorov prediction formula.
In the case of an AR(p) process the prediction formula can also be written as
(s) (s) (s)
E[Yt+s |Yt , Yt−1 , ...] = µ + f11 (Yt − µ) + f12 (Yt−1 − µ) + ... + f1p (Yt−p+1 − µ)
(j)
where f11 is the element (1, 1) of the matrix F j .
The easiest way to compute prediction from AR(p) model is, however, through recursive
methods.

Prediction
write the model in its AR(∞) representation:

η(L)(Yt − µ) = ǫt
then the optimal prediction of Yt+s is given by

η(L)−1 h i
E[Yt+s |Yt , Yt−1 , ...] = µ + η(L)(Yt − µ) with Lk = 0 for k < 0
Ls + +
which is known as Wiener-Kolmogorov prediction formula.
In the case of an AR(p) process the prediction formula can also be written as
(s) (s) (s)
E[Yt+s |Yt , Yt−1 , ...] = µ + f11 (Yt − µ) + f12 (Yt−1 − µ) + ... + f1p (Yt−p+1 − µ)
(j)
where f11 is the element (1, 1) of the matrix F j .
The easiest way to compute prediction from AR(p) model is, however, through recursive
methods.

Box-Jenkins Approach
check for stationarity: if not try different transformation (ex differentiation→ARIMA models)
Identification:
check the autocorrelation (ACF) function: a q-correlated process is an MA(q) model
check the partial autocorrelation (PACF) function:
for an AR(p) process, while the k–lag ACF can be interpreted as simple regression
Yt = ρ(k)Yt−k + error, the k–lag PACF can be seen as a multiple regression
Yt = b1 Yt−1 + b2 Yt−2 + ... + bk Yt−k + error
it can be computed by solving the Yule-Walker system:
    −1  
b1 γ(0) γ(1) ... γ(k − 1) γ(1)
 b2   γ(1) γ(0) ... γ(k − 2)   γ(2) 
     
 .  =  . . .   . 
 .   . . .   . 
. . . ... . .
bk γ(k − 1) γ(k − 2) ... γ(0) γ(k)
Importantly, AR(p) processes are “p–partially correlated” ⇒ identification of AR order
Validation: check the appropriateness of the model by some measure of fit.

AIC/Akaike = T log σ̂e2 + 2 m
BIC/Schwarz = T log σ̂e2 + m log T
with σe2 estimation error variance, m = p + q + 1 n◦ of parameters, and T n◦ of obs
Diagnostic checking of the residuals.
Identification:
    −1  
b1 γ(0) γ(1) ... γ(k − 1) γ(1)
 b2   γ(1) γ(0) ... γ(k − 2)   γ(2) 
     
 .  =  . . .   . 
 .   . . .   . 
. . . ... . .
bk γ(k − 1) γ(k − 2) ... γ(0) γ(k)

Identification:
    −1  
b1 γ(0) γ(1) ... γ(k − 1) γ(1)
 b2   γ(1) γ(0) ... γ(k − 2)   γ(2) 
     
 .  =  . . .   . 
 .   . . .   . 
. . . ... . .
bk γ(k − 1) γ(k − 2) ... γ(0) γ(k)

ARIMA
Integrated ARMA model:
ARIMA(p,1,q) denote a nonstationary process Yt for which the first difference

Yt − Yt−1 = (1 − L)Yt is a stationary ARMA(p,q) process.
⇓
Yt is said to be integrated of order 1 or I(1).
If 2 differentiations of Yt are necessary to get a stationary process i.e.

(1 − L)2 Yt
⇓
then the process Yt is said to be integrated of order 2 or I(2).
I(0) indicate a stationary process.

ARIMA
Integrated ARMA model:
ARIMA(p,1,q) denote a nonstationary process Yt for which the first difference

Yt − Yt−1 = (1 − L)Yt is a stationary ARMA(p,q) process.
⇓
Yt is said to be integrated of order 1 or I(1).
If 2 differentiations of Yt are necessary to get a stationary process i.e.

(1 − L)2 Yt
⇓
then the process Yt is said to be integrated of order 2 or I(2).
I(0) indicate a stationary process.

ARFIMA
The k-difference operator (1 − L)n with integer n can be generalized to a fractional

difference operator (1 − L)d with 0 < d < 1 defined by the binomial expansion
(1 − L)d = 1 − dL + d(d − 1)L2 /2! − d(d − 1)(d − 2)L3 /3! + ...
obtaining a fractionally integrated process of order d i.e. I(d).
If d < 0.5 the process is cov stationary and admits an AR(∞) representation.
The usefulness of a fractional filter (1 − L)d is that it produces hyperbolic decaying

autocorrelations i.e. the so called long memory. In fact, for ARFIMA(p,d,q) processes
φ(L)(1 − L)d Yt = θ(L)ǫt
the autocorrelation functions is proportional to
ρ(k) ≈ ck2d−1

ARFIMA
The k-difference operator (1 − L)n with integer n can be generalized to a fractional

difference operator (1 − L)d with 0 < d < 1 defined by the binomial expansion
(1 − L)d = 1 − dL + d(d − 1)L2 /2! − d(d − 1)(d − 2)L3 /3! + ...
obtaining a fractionally integrated process of order d i.e. I(d).
If d < 0.5 the process is cov stationary and admits an AR(∞) representation.
The usefulness of a fractional filter (1 − L)d is that it produces hyperbolic decaying

autocorrelations i.e. the so called long memory. In fact, for ARFIMA(p,d,q) processes
φ(L)(1 − L)d Yt = θ(L)ǫt
the autocorrelation functions is proportional to
ρ(k) ≈ ck2d−1

ARCH and GARCH models

Basic Structure and Properties of ARMA model
standard time series models have:

Yt = E[Yt |Ωt−1 ] + ǫt
E[Yt |Ωt−1 ] = f (Ωt−1 ; θ)
h i
Var [Yt |Ωt−1 ] = E ǫ2t |Ωt−1 = σ 2
hence,
Conditional mean: varies with Ωt−1

Conditional variance: constant (unfortunately)
k-step-ahead mean forecasts: generally depends on Ωt−1

k-step-ahead variance forecasts : depends only on k, not on Ωt−1 (again
unfortunately)
Unconditional mean: constant

Unconditional variance: constant

AutoRegressive Conditional Heteroskedasticity
(ARCH) model
Engle (1982, Econometrica) intruduced the ARCH models:

Yt = E[Yt |Ωt−1 ] + ǫt
E[Yt |Ωt−1 ] = f (Ωt−1 ; θ)
h i
Var [Yt |Ωt−1 ] = E ǫ2t |Ωt−1 = σ (Ωt−1 ; θ) ≡ σt2
hence,
Conditional mean: varies with Ωt−1

Conditional variance: varies with Ωt−1
k-step-ahead mean forecasts: generally depends on Ωt−1

k-step-ahead variance forecasts : generally depends on Ωt−1
Unconditional mean: constant

Unconditional variance: constant

ARCH(q)

How to parameterize E ǫ2t |Ωt−1 = σ((Ωt−1 ; θ) ≡ σt2 ?
ARCH(q) postulated that the conditional variance is a linear function of the past q squared
innovations
Xq
σt2 = ω + αi ǫ2t−i = ω + α(L)ǫ2t−1
i=1
Defining vt = ǫ2t − σt2 , the ARCH(q) model can be written as

ǫ2t = ω + α(L)ǫ2t−1 + vt
Since Et−1 (vt ) = 0, the model corresponds directly to an AR(q) model for the squared
innovations, ǫ2t .
The process is covariance stationary if and only if the sum of the positive AR parameters is
less than 1. Then, the unconditional variance of ǫt is
Var(ǫt ) = σ 2 = w/(1 − α1 − α2 − ... − αq ).

ARCH(q)

How to parameterize E ǫ2t |Ωt−1 = σ((Ωt−1 ; θ) ≡ σt2 ?
ARCH(q) postulated that the conditional variance is a linear function of the past q squared
innovations
Xq
σt2 = ω + αi ǫ2t−i = ω + α(L)ǫ2t−1
i=1
Defining vt = ǫ2t − σt2 , the ARCH(q) model can be written as

ǫ2t = ω + α(L)ǫ2t−1 + vt
Since Et−1 (vt ) = 0, the model corresponds directly to an AR(q) model for the squared
innovations, ǫ2t .
The process is covariance stationary if and only if the sum of the positive AR parameters is
less than 1. Then, the unconditional variance of ǫt is
Var(ǫt ) = σ 2 = w/(1 − α1 − α2 − ... − αq ).

AR(1)-ARCH(1)
Example: the AR(1)-ARCH(1) model
Yt = φYt−1 + ǫt
σt2 = ω + αǫ2t−1
ǫt ∼ N(0, σt2 )
- Conditional mean: E(Yt |Ωt−1 ) = φYt−1

- Conditional variance: E([Yt − E(Yt |Ωt−1 )]2 |Ωt−1 = ω + αǫ2t−1
- Unconditional mean: E(Yt ) = 0
1 ω
- Unconditional variance: E(Yt − E(Yt ))2 = (1−φ 2 ) (1−α)
Note that the unconditional distribution of ǫt has Fat Tail.

In fact, the unconditional kurtosis E(ǫ4t )/E(ǫ2t )2 is
h i h i h i
E ǫ4t = E E(ǫ4t |Ωt−1 ) = 3E σt4 = 3[Var(σt2 ) + E(σt2 )2 ] = 3[Var(σt2 ) +E(ǫ2t )2 ] > 3E(ǫ2t )2 .
| {z }
>0
Hence,
Kurtosis(ǫt ) = E(ǫ4t )/E(ǫ2t )2 > 3

AR(1)-ARCH(1)
Example: the AR(1)-ARCH(1) model
Yt = φYt−1 + ǫt
σt2 = ω + αǫ2t−1
ǫt ∼ N(0, σt2 )
- Conditional mean: E(Yt |Ωt−1 ) = φYt−1

- Conditional variance: E([Yt − E(Yt |Ωt−1 )]2 |Ωt−1 = ω + αǫ2t−1
- Unconditional mean: E(Yt ) = 0
1 ω
- Unconditional variance: E(Yt − E(Yt ))2 = (1−φ 2 ) (1−α)
Note that the unconditional distribution of ǫt has Fat Tail.

In fact, the unconditional kurtosis E(ǫ4t )/E(ǫ2t )2 is
h i h i h i
E ǫ4t = E E(ǫ4t |Ωt−1 ) = 3E σt4 = 3[Var(σt2 ) + E(σt2 )2 ] = 3[Var(σt2 ) +E(ǫ2t )2 ] > 3E(ǫ2t )2 .
| {z }
>0
Hence,
Kurtosis(ǫt ) = E(ǫ4t )/E(ǫ2t )2 > 3

GARCH(p,q)
Problem: empirical volatility very persistent ⇒ Large q i.e. too many α’s
Bollerslev (1986, J. of Econometrics) proposed the Generalized ARCH model.
The GARCH(p,q) is defined as
q
X p
X
σt2 = ω + αi ǫ2t−i + 2
βj σt−j = ω + α(L)ǫ2t−1 + β(L)σt−1
2
i=1 j=1
As before also the GARCH(p,q) can be rewritten as
ǫ2t = ω + [α(L) + β(L)] ǫ2t−1 − β(L)vt−1 + vt
which defines an ARMA[max(p, q),p] model for et z.

GARCH(p,q)
Problem: empirical volatility very persistent ⇒ Large q i.e. too many α’s
Bollerslev (1986, J. of Econometrics) proposed the Generalized ARCH model.
The GARCH(p,q) is defined as
q
X p
X
σt2 = ω + αi ǫ2t−i + 2
βj σt−j = ω + α(L)ǫ2t−1 + β(L)σt−1
2
i=1 j=1
As before also the GARCH(p,q) can be rewritten as
ǫ2t = ω + [α(L) + β(L)] ǫ2t−1 − β(L)vt−1 + vt
which defines an ARMA[max(p, q),p] model for et z.

GARCH(1,1)
By far the most commonly used is the GARCH(1,1):
σt2 = ω + α ǫ2t−1 + βj σt−1
2
with ω > 0, α > 0, β > 0.
By recursive substitution, the GARCH(1,1) may be written as the following ARCH(∞):

∞
X
σt2 = ω(1 − β) + α β i−1 ǫ2t−i
i=1
which reduces to an exponentially weighted moving average filter for ω = 0 and α + β = 1

(sometimes referred to as Integrated GARCH or IGARCH(1,1)).
Moreover, GARCH(1,1) implies an ARMA(1,1) representation in the ǫ2t

ǫ2t = ω + [α + β]ǫ2t−1 − βvt−1 + vt
Forecasting. Denoting the unconditional variance σ 2 ≡ ω(1 − α − β)−1 we have:

2
σ̂t+h|t = σ 2 + (α + β)h−1 (σt+1
2
− σ2 )
showing that the forecasts of the conditional variance revert to the long-run unconditional
variance at an exponential rate dictated by α + β

GARCH(1,1)
By far the most commonly used is the GARCH(1,1):
σt2 = ω + α ǫ2t−1 + βj σt−1
2
with ω > 0, α > 0, β > 0.
By recursive substitution, the GARCH(1,1) may be written as the following ARCH(∞):

∞
X
σt2 = ω(1 − β) + α β i−1 ǫ2t−i
i=1
which reduces to an exponentially weighted moving average filter for ω = 0 and α + β = 1

(sometimes referred to as Integrated GARCH or IGARCH(1,1)).
Moreover, GARCH(1,1) implies an ARMA(1,1) representation in the ǫ2t

ǫ2t = ω + [α + β]ǫ2t−1 − βvt−1 + vt
Forecasting. Denoting the unconditional variance σ 2 ≡ ω(1 − α − β)−1 we have:

2
σ̂t+h|t = σ 2 + (α + β)h−1 (σt+1
2
− σ2 )
showing that the forecasts of the conditional variance revert to the long-run unconditional
variance at an exponential rate dictated by α + β

Asymmetric GARCH
News Impact Curve
8
symmetric GARCH
asymmetric GARCH
In standard GARCH model:
conditional variance
6
σt2 =ω+ 2
αrt−1 + 2
βσt−1
4
σt2 responds symmetrically to past returns.

2
The so called “news impact curve” is a parabola

0
−5 0 5
standardized lagged shocks
Empirically negative rt−1 impact more than positive ones → asymmetric news impact curve
GJR or T-GARCH

1 if rt < 0
σt2 = 2
ω + αrt−1 2
+ γrt−1 2
Dt−1 + βσt−1 with Dt =
0 otherwise
- Positive returns (good news): α
- Negative returns (bad news): α + γ
- Empirically γ > 0 → “Leverage effect”
Exponential GARCH (EGARCH)

rt−1
ln(σt2 ) = ω + α + γ rt−1 + βln(σ 2 )
t−1
σt−1 σt−1

Asymmetric GARCH
News Impact Curve
8
symmetric GARCH
asymmetric GARCH
In standard GARCH model:
conditional variance
6
σt2 =ω+ 2
αrt−1 + 2
βσt−1
4
σt2 responds symmetrically to past returns.

2
The so called “news impact curve” is a parabola

0
−5 0 5
standardized lagged shocks
Empirically negative rt−1 impact more than positive ones → asymmetric news impact curve
GJR or T-GARCH

1 if rt < 0
σt2 = 2
ω + αrt−1 2
+ γrt−1 2
Dt−1 + βσt−1 with Dt =
0 otherwise
- Positive returns (good news): α
- Negative returns (bad news): α + γ
- Empirically γ > 0 → “Leverage effect”
Exponential GARCH (EGARCH)

rt−1
ln(σt2 ) = ω + α + γ rt−1 + βln(σ 2 )
t−1
σt−1 σt−1

Estimation
A GARCH process with gaussian innovation:

rt |Ωt−1 ∼ N(µt (θ), σt2 (θ))
has conditional densities:

1 1 (rt − µ(θ))
f (rt |Ωt−1 ; θ) = √ σt−1 (θ) exp − 2
2π 2 σt (θ)
using the “prediction–error” decomposition of the likelihood
L(rT , rT−1 , ..., r1 ; θ) = f (rT |ΩT−1 ; θ) × f (rT−1 |ΩT−2 ; θ) × ... × f (r1 |Ω0 ; θ)
the log-likelihood becomes:

T T
T X 1 X (rt − µ(θ))
log L(rT , rT−1 , ..., r1 ; θ) = − log(2π) − log σt (θ) −
2 t=1
2 t=1 σt2 (θ)
Non–linear function in θ ⇒ Numerical optimization techiques.


ARIMA and GARCH

Enviado por

Dados do documento

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

ARIMA and GARCH

Enviado por

Direitos autorais:

Formatos disponíveis

Introduction to ARMA and GARCH processes

Fulvio Corsi Introduction to ARMA

Fulvio Corsi Introduction to ARMA

Fulvio Corsi Introduction to ARMA

Fulvio Corsi Introduction to ARMA

Fulvio Corsi Introduction to ARMA

and admits power exponent, for instance:

Fulvio Corsi Introduction to ARMA

and admits power exponent, for instance:

Fulvio Corsi Introduction to ARMA

and admits power exponent, for instance:

Fulvio Corsi Introduction to ARMA

and admits power exponent, for instance:

Fulvio Corsi Introduction to ARMA

hence, while a withe noise is “0-correlated”, MA(1) is 1-correlated

hence, while a withe noise is “0-correlated”, MA(1) is 1-correlated

Fulvio Corsi Introduction to ARMA

Fulvio Corsi Introduction to ARMA

Fulvio Corsi Introduction to ARMA

If an MA process can be written as an AR(∞) of this type, such MA representation is said to

For a general MA(q) process

If an MA process can be written as an AR(∞) of this type, such MA representation is said to

For a general MA(q) process

The eigenvalues of F are given by solving the characteristic equation

Fulvio Corsi Introduction to ARMA

The eigenvalues of F are given by solving the characteristic equation

Fulvio Corsi Introduction to ARMA

An ARMA(p,q) process is defined as

the process is stationary if all the roots of

the process is invertible if all the roots of

Fulvio Corsi Introduction to ARMA

Yt = φ1 Yt−1 + φ2 Yt−2 + ... + φp Yt−p + ǫt

OLS are consistent and under gaussianity asymptotically equivalent to MLE

For example, the full Likelihood of an AR(1) can be written as

For a general ARMA(p,q)

Yt−1 is correlated with ǫt−1 , ..., ǫt−q ⇒ OLS not consistent.

→ MLE with numerical optimization procedures.

Fulvio Corsi Introduction to ARMA

Yt = φ1 Yt−1 + φ2 Yt−2 + ... + φp Yt−p + ǫt

OLS are consistent and under gaussianity asymptotically equivalent to MLE

For example, the full Likelihood of an AR(1) can be written as

For a general ARMA(p,q)

Yt−1 is correlated with ǫt−1 , ..., ǫt−q ⇒ OLS not consistent.

→ MLE with numerical optimization procedures.

Fulvio Corsi Introduction to ARMA

write the model in its AR(∞) representation:

then the optimal prediction of Yt+s is given by

which is known as Wiener-Kolmogorov prediction formula.

Fulvio Corsi Introduction to ARMA

write the model in its AR(∞) representation:

then the optimal prediction of Yt+s is given by

which is known as Wiener-Kolmogorov prediction formula.

Fulvio Corsi Introduction to ARMA

Importantly, AR(p) processes are “p–partially correlated” ⇒ identification of AR order

Validation: check the appropriateness of the model by some measure of fit.

Importantly, AR(p) processes are “p–partially correlated” ⇒ identification of AR order

Validation: check the appropriateness of the model by some measure of fit.

Importantly, AR(p) processes are “p–partially correlated” ⇒ identification of AR order

Validation: check the appropriateness of the model by some measure of fit.

Integrated ARMA model:

ARIMA(p,1,q) denote a nonstationary process Yt for which the first difference

If 2 differentiations of Yt are necessary to get a stationary process i.e.

I(0) indicate a stationary process.

Fulvio Corsi Introduction to ARMA