ECON 203C: System Models

TA Note 2: Version 2
Seemingly Unrelated Regression (SUR) Model
Hisayuki Yoshimoto
Last Modied: April 15, 2008
Abstract: In section 1, we review concepts of seemingly unrelated (SUR) model and SUR (or equivalently GLS)
estimator. In section 2, we solve Final Review Question 5 which consists of very fundamental problems. In section
3, We discuss Final 2006 Question 2 that requires skills of matrix algebra manipulation, especially Kronecker product
manipulation. In Section 4, we deal with matrix algebra intensive problem, Comp 2003F Question 1. In Section 5, we
solve Comp 2006F Question1.
1 Seemingly Unrelated Regression Model
In short, seemingly uncorrelated regression (SUR) estimator is nothing more than GLS estimator. More precisely, it is
gigantic GLS, gigantic in the sense of matrix dimension. Since GLS estimator is ecient, SUR (or equivalently GLS)
estimator has the advantage of eciency (i.e. small variance) compared to OLS estimator. However, since we have two
indices, individual index and equation index, special attention is required when we stack up data and construct data
matrix, 1, A, and l.
1.1 Model
There are individuals (agents) in the data and each of individuals has J equations. Dene i = 1, . . . , as individual
index and , = 1, . . . , J as equation index. An individual i has J equations,
j
1i
..
11
= r
0
1i
..
1k1
,
1
..
k11
+ n
1i
..
11
(individual is 1st equation)
.
.
.
j
ji
..
11
= r
0
ji
..
1kj
,
j
..
kj1
+ n
ji
..
11
(individual is ,-th equation)
.
.
.
j
Ji
..
11
= r
0
Ji
..
1k
J
,
J
..
k
J
1
+ n
Ji
..
11
(individual is ,-th equation)
Here we assume the conditional expectation and variance-covariance of individual is error terms (n
1i
, , n
ji
, , n
Ji
)
is
1
_

_
_

_
n
1i
n
2i
.
.
.
n
ji
.
.
.
n
Ji
_

all r
ji
for , = 1, , J, i = 1, . . .
_

_
=
_

_
0
11
0
11
.
.
.
0
11
.
.
.
0
11
_

_
= 0
J1
1
and

..
JJ
= \ ar
_

_
_

_
n
1i
n
2i
.
.
.
n
ji
.
.
.
n
Ji
_

all r
ji
for , = 1, , J, i = 1, . . .
_

_
=
_

_
o
11
o
12
o
1j
o
1J
o
21
o
22
o
2j
o
2j
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
o
j1
o
j2
o
jj
o
jJ
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
o
J1
o
J2
o
Jj
o
JJ
_

_
.
Note that o-diagonal elements of above conditional variance-covariance matrix are non-zero. In other words, individual
is errors, (n
1i
, , n
ji
, , n
Ji
) are correlated. For example, if individuals errors are positively correlated, then an
individual i who has large error term in ,-th equation (high n
ji
) is likely to have large large error term in ,
0
th (,
0
,= ,)
equation (high n
j
0
i
).
However, we also assume that for individual is errors and individual i
0
(,= i)s errors are
Co
_

_
_

_
n
1i
n
2i
.
.
.
n
ji
.
.
.
n
Ji
_

_
. .
individual is error terms
,
_

_
n
1i
0
n
2i
0
.
.
.
n
ji
0
.
.
.
n
Ji
0
_

_
. .
individual i
0
s error terms

all r
ji
for , = 1, , J, i = 1, . . .
_

_
=
_

_
0
11
0
11
0
11
0
11
0
11
0
11
0
11
0
11
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0
11
0
11
0
11
0
11
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0
11
0
11
0
11
0
11
_

_
= O
JJ
.
i.e. individual is error vector (n
1i
, . . . , n
ji
, . . . , n
Ji
) and individual i
0
s error vector (n
1i
0 , . . . , n
ji
0 , . . . , n
Ji
0 ) are not
correlated (errors are uncorrelated across individuals). In summary, error terms are correlated within individual, but
uncorrelated across individuals.
Now, our goal is eciently estimating ,
1
, . . . , ,
j
, . . . ,
J
by exploiting the information of correlation of (within individ-
ual) error terms. We stack up variables as follows. Notice that we stack up variables by grouping equations (grouping by
equation index). For equation , = 1, . . . , J, we stack up and construct equation vectors and matrix.
j
j
..
N1
=
_

_
j
j1
.
.
.
j
ji
.
.
.
j
jN
_

_
=
_

_
j
j1
..
11
.
.
.
j
ji
..
11
.
.
.
j
jJ
..
11
_

_
(stacking up dependent variables of ,-th equation),
A
j
..
Nkj
=
_

_
r
0
j1
.
.
.
r
0
ji
.
.
.
r
0
jN
_

_
=
_

_
r
0
j1
..
1kj
.
.
.
r
0
ji
..
1kj
.
.
.
r
0
jN
..
1kj
_

_
(stacking up independent variables of ,-th equation),
2
and
n
j
..
N1
=
_

_
n
j1
.
.
.
n
ji
.
.
.
n
jN
_

_
=
_

_
n
j1
..
11
.
.
.
n
ji
..
11
.
.
.
n
jN
..
11
_

_
(stacking up error terms of ,-th equation).
Then, we further stack up
1
..
NJ1
=
_

_
j
1
.
.
.
j
j
.
.
.
j
J
_

_
=
_

_
j
1
..
N1
.
.
.
j
j
..
N1
.
.
.
j
J
..
N1
_

_
, (stacking up dependent vectors of all equation),
A
..
NJ

j=1
kj
!
=
_

_
A
1
..
Nk1
O
Nk2
O
Nkj
O
Nk
J
O
Nk1
A
2
..
Nk2
O
Nkj
O
Nk
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
O
Nk1
O
Nk2
A
j
..
Nkj
O
Nk
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
O
Nk1
O
Nk2
O
Nkj
A
J
..
Nk
J
_

_
(stacking up independent variable matrices of all equation),
and
l
..
NJ1
=
_

_
n
1
.
.
.
n
j
.
.
.
n
J
_

_
=
_

_
n
1
..
N1
.
.
.
n
j
..
N1
.
.
.
n
J
..
N1
_

_
, (stacking up error vectors of all equations).
We also stack up parameters (that we are trying to estimate) too
,
..

j=1
kj
!
1
=
_

_
,
1
.
.
.
,
j
.
.
.
,
J
_

_
=
_

_
,
1
..
k11
.
.
.
,
j
..
kj1
.
.
.
,
J
..
k
J
1
_

_
. (stacking up parameter vectors of all equations)
3
Then, the model can be written as
1 = A, +l
1
..
NJ1
= A
..
NJ

j=1
kj
!
,
..

j=1
kj
!
1
+ l
..
NJ1
.
Remind that the elements and dimensionality of above model equation are
_

_
j
1
..
N1
j
2
..
N1
.
.
.
j
j
..
N1
.
.
.
j
J
..
N1
_

_
. .
NJ1
=
_

_
A
1
..
Nk1
O
Nk2
O
Nkj
O
Nk
J
O
Nk1
A
2
..
Nk2
O
Nkj
O
Nk
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
O
Nk1
O
Nk2
A
j
..
Nkj
O
Nk
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
O
Nk1
O
Nk2
O
Nkj
A
J
..
Nk
J
_

_
. .
NJ

j=1
kj
!
_

_
,
1
..
k11
,
2
..
k21
.
.
.
,
j
..
kj1
.
.
.
,
J
..
k
J
1
_

_
. .

j=1
kj
!
1
+
_

_
n
1
..
N1
n
2
..
N1
.
.
.
n
j
..
N1
.
.
.
n
J
..
N1
_

_
. .
NJ1
=
_

_
A
1
..
Nk1
,
1
..
k11
+ n
1
..
N1
A
2
..
Nk2
,
2
..
k21
+ n
2
..
N1
.
.
.
A
j
..
Nkj
,
j
..
kj1
+ n
j
..
N1
.
.
.
A
J
..
Nk
J
,
J
..
k
J
1
+ n
J
..
N1
_

_
1.2 Equation by Equation OLS
We have the model (rewriting)
1 = A, +l (1)
1
..
NJ1
= A
..
NJ

j=1
kj
!
,
..

j=1
kj
!
1
+ l
..
NJ1
.
The inecient (but easy) method for estimating , is just applying OLS to equation (1). Then, we get the OLS
estimator
^
,
OLS
= (A
0
A)
1
A
0
1
^
,
OLS
. .

j=1
kj
!
1
=
_
_
_
_
_
_
_
A
0
..

j=1
kj
!
NJ
A
..
NJ

j=1
kj
!
_
_
_
_
_
_
_
1
A
0
..

j=1
kj
!
NJ
1
..
NJ1
4
Decomposing
^
,
OLS
estimator
^
,
OLS
= (A
0
A)
1
A
0
1
=
_
_
_
_
_
_
_
_
_
_
_

_
A
1
O
Nk2
O
Nkj
O
Nk
J
O
Nk1
A
2
O
Nkj
O
Nk
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
O
Nk1
O
Nk2
A
j
O
Nk
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
O
Nk1
O
Nk2
O
Nkj
A
J
_

_
0
_

_
A
1
O
Nk2
O
Nkj
O
Nk
J
O
Nk1
A
2
O
Nkj
O
Nk
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
O
Nk1
O
Nk2
A
j
O
Nk
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
O
Nk1
O
Nk2
O
Nkj
A
J
_

_
_
_
_
_
_
_
_
_
_
_
1

_
A
1
O
Nk2
O
Nkj
O
Nk
J
O
Nk1
A
2
O
Nkj
O
Nk
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
O
Nk1
O
Nk2
A
j
O
Nk
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
O
Nk1
O
Nk2
O
Nkj
A
J
_

_
0
_

_
j
1
j
2
.
.
.
j
j
.
.
.
j
J
_

_
=
_

_
_

_
(A
0
1
A
1
)
1
O
k1k2
O
k1kj
O
k1k
J
O
k2k1
(A
0
2
A
2
)
1
O
k2kj
O
k2k
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
O
kjk1
O
kjk2

_
A
0
j
A
j
_
1
O
kjk
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
O
k
J
k1
O
k
J
k2
O
k
J
kj
(A
0
J
A
j
)
1
_

_
_

_
_

_
A
0
1
j
1
A
0
2
j
2
.
.
.
A
0
j
j
j
.
.
.
A
0
J
j
J
_

_
=
_

_
(A
0
1
A
1
)
1
A
0
1
j
1
(A
0
2
A
2
)
1
A
0
2
j
2
.
.
.
_
A
0
j
A
j
_
1
A
0
j
j
j
.
.
.
(A
0
J
A
j
)
1
A
0
J
j
J
_

_
See foot note
1
.This estimator is equivalent to applying OLS to each equation. Note that
^
,
OLS
is inecient but con-
sistent estimator with large sample, because conditional expectation of error vector is zero vector. In the next subsection,
we derive more ecient estimator by applying GLS method.
1.3 Seemingly Unrelated Regression
We have the model (rewriting)
1 = A, +l (2)
1
..
NJ1
= A
..
NJ

j=1
kj
!
,
..

j=1
kj
!
1
+ l
..
NJ1
.
The ecient method for estimating , is applying GLS to model equation (2). To apply GLS, we rst need to derive
variance-covariance matrix of error vector l. That is (all variances and expectations are conditioned by A, for notational
1
In this giantic matrix calculation, I skipped the detail between second and third equation. In sectino 3 (Final 2006 Question 2), we discuss
the same calculation with the number of equation J = 3: So, please refer secition 3 for detail matrix operations.
5
simplicity I omit writing conditional terms)
\ ar [l]
. .
NJNJ
= \ ar
_

_
_

_
n
1
..
N1
n
2
..
N1
.
.
.
n
j
..
N1
.
.
.
n
J
..
N1
_

_
_

_
= 1
_

_
_
_
_
_
_
_
_
_
_
_
_

_
n
1
n
2
.
.
.
n
j
.
.
.
n
J
_

_
0
NT1
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_

_
n
1
n
2
.
.
.
n
j
.
.
.
n
J
_

_
0
NT1
_
_
_
_
_
_
_
_
_
_
0
_

_
(since 1
_

_
_

_
n
1
n
2
.
.
.
n
j
.
.
.
n
J
_

_
_

_
= 0
NT1
)
= 1
_

_
_

_
n
1
n
0
1
n
1
n
0
2
n
1
n
0
j
n
1
n
0
J
n
2
n
0
1
n
1
n
0
2
n
2
n
0
j
n
2
n
0
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
n
j
n
0
1
n
j
n
0
2
n
j
n
0
j
n
j
n
0
J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
n
J
n
0
1
n
J
n
0
2
n
J
n
0
j
n
J
n
0
J
_

_
_

_
=
_

_
1 [n
1
n
0
1
] 1 [n
1
n
0
2
] 1
_
n
1
n
0
j

1 [n
1
n
0
J
]
1 [n
2
n
0
1
] 1 [n
1
n
0
2
] 1
_
n
2
n
0
j

1 [n
2
n
0
J
]
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1 [n
j
n
0
1
] 1 [n
j
n
0
2
] 1
_
n
j
n
0
j

1 [n
j
n
0
J
]
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1 [n
J
n
0
1
] 1 [n
J
n
0
2
] 1
_
n
J
n
0
j

1 [n
J
n
0
J
]
_

_
=
_

_
o
11
1
n
o
12
1
n
o
1j
1
n
o
1J
1
n
o
21
1
n
o
22
1
n
o
2j
1
n
o
2J
1
n
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
o
j1
1
n
o
j2
1
n
o
jj
1
n
o
jJ
1
n
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
o
J1
1
n
o
J2
1
n
o
Jj
1
n
o
JJ
1
n
_

_
(see detail below)
=
_

_
o
11
o
12
o
1j
o
1J
o
21
o
22
o
2j
o
2J
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
o
j1
o
j2
o
jj
o
jJ
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
o
J1
o
J2
o
Jj
o
JJ
_

_
1
n
= 1
n
.
See footnote for denition of Kronecker product
2
. Note that in the above calculation we used the fact that for equation
, and equation ,
0
(,= ,), conditional expectation of product of error vectors is (omitting conditional terms for simplicity
2
Denition: Kronecker Product
If
A
|{z}
lm
=
2
6
4
a
11
a
1m
.
.
.
.
.
.
.
.
.
a
l1
a
lm
3
7
5 and B
|{z}
no
=
2
6
4
b
11
b
1o
.
.
.
.
.
.
.
.
.
b
n1
bno
3
7
5
then
A B
| {z }
lnmo
=
2
6
4
a
11
a
1m
.
.
.
.
.
.
.
.
.
a
l1
a
lm
3
7
5 B =
2
6
4
a
11
B a
1m
B
.
.
.
.
.
.
.
.
.
a
l1
B a
lm
B
3
7
5
For example, if
A =

1 2
3 4

and B =

5 6
7 8

;
Then,
A B =
2
6
6
4
1

5 6
7 8

5 6
7 8

5 6
7 8

5 6
7 8

3
7
7
5
=
2
6
6
4
5 6 10 12
7 8 14 16
15 18 20 24
21 24 28 32
3
7
7
5
:
6
of notation)
1
_
n
j
n
0
j
0

. .
NN
= 1
_

_
_

_
n
j1
n
j2
.
.
.
n
JN
_

_
. .
N1
_
n
j
0
1
n
j
0
2
n
j
0
N

. .
1N
_

_
= 1
_

_
_

_
n
j1
n
j
0
1
n
j1
n
j
0
2
n
j
n
j
0
N
n
j2
n
j
0
1
n
j2
n
j
0
2
n
j2
n
j
0
N
.
.
.
.
.
.
.
.
.
.
.
.
n
JN
, n
j
0
1
n
JN
n
j
0
2
n
JN
n
j
0
N
_

_
_

_
=
_

_
1 [n
j1
n
j
0
1
] 1 [n
j1
n
j
0
2
] 1 [n
j1
n
j
0
N
]
1 [n
j2
n
j
0
1
] 1 [n
j2
n
j
0
2
] 1 [n
j2
n
j
0
N
]
.
.
.
.
.
.
.
.
.
.
.
.
1 [n
JN
n
j
0
1
] 1 [n
JN
n
j
0
2
] 1 [n
JN
n
j
0
N
]
_

_
=
_

_
o
jj
0 0
11
0
11
0
11
o
jj
0 0
11
.
.
.
.
.
.
.
.
.
.
.
.
0
11
0
11
o
jj
0
_

_
= o
jj
0 1
N
,
because errors are uncorrelated across individuals.
Thus, conditional variance-covariance matrix of l is
\ ar [ l[ A]
. .
NJNJ
= 1
N
=
..
JJ
1
n
..
NN
and GLS estimator of the model (we call it seemingly unrelated regression estimator
^
,
SUR
or equivalently GLS
estimator
^
,
GLS
) is
^
,
SUR
=
^
,
GLS
=
_
A
0
( 1
n
)
1
A
_
A
0
( 1
n
)
1
1
^
,
SUR
. .

j=1
kj
!
1
=
_
_
_
_
_
_
_
A
0
..

j=1
kj
!
NJ
( 1
n
)
1
. .
NJNJ
A
..
NJ

j=1
kj
!
_
_
_
_
_
_
_
1
A
0
..

j=1
kj
!
NJ
( 1
n
)
1
. .
NJNJ
1
..
NJ1
In practice, feasible seemingly unrelated estimator
^
,
FSUR
(or equivalently, feasible GLS estimator
^
,
FGLS
) is obtained
by three steps
Step 1:
Implement equation by equation OLS to the model (2) and obtain
^
,
OLS
= (A
0
A)
1
A
0
1.
Calculate residual vector
^
l
..
NJ1
= 1 A
^
,
OLS
.
Remind that
^
,
OLS
is inecient but consistent estimator.
7
Note that
^
l consists of (I am writing details here to avoid confusion)
^
l
..
NJ1
=
_

_
^ n
1
..
N1
.
.
.
^ n
j
..
N1
.
.
.
^ n
J
..
N1
_

_
=
_

_
[1st equations residual vector]
.
.
.
[,-th equations residual vector]
.
.
.
[J-th equations residual vector]
_

_
=
_

_
_

_
^ n
11
^ n
12
.
.
.
^ n
1N
_

_
.
.
.
_

_
^ n
j1
^ n
j2
.
.
.
^ n
jN
_

_
.
.
.
_

_
^ n
J1
^ n
J2
.
.
.
^ n
JN
_

_
_

_
=
_

_
_

_
1st equations individual 1s residual
1st equations individual 2s residual
.
.
.
1st equations individual s residual
_

_
.
.
.
_

_
2nd equations individual 1s residual
2nd equations individual 2s residual
.
.
.
2nd equations individual s residual
_

_
.
.
.
_

_
-th equations individual 1s residual
-th equations individual 2s residual
.
.
.
-th equations individual s residual
_

_
_

_
.
(3)
Step 2:
Based on
^
l calculated in step 1, obtain the estimate of by
^

..
JJ
=
1
:
N

i=1
_

_
^ n
1i
.
.
.
^ n
ji
.
.
.
^ n
Ji
_

_
. .
J1
_
^ n
1i
^ n
ji
^ n
Ji

. .
1J
=
1
:
N

i=1
(individual i
0
s residual vector) (individual i
0
s residual vector)
0
(4)
Notice that the summation is indexed by i (individual index) and error vector in equation (4) is dierent from error
vector of (3).
Step 3:
By utilizing
^
that we have derived in step 2, estimate feasible seemingly unrelated regression coecient by
^
,
FSUR
=
^
,
FGLS
=
_
A
0
_
^
1
n
_
1
A
_
A
0
_
^
1
n
_
1
1
1.4 Eciency of SUR (GLS) Estimator
The SUR (GLS) estimator is ecient due to the Gauss-Markov Theorem.
Formally, the model equation
1 = A, +l
has the conditional variance of error vector
\ ar [ l[ A] = 1
n
.
Decomposing the inverse of conditional expectation as follows
( 1
n
)
1
= 1
0
1,
where 1 is J J nonsingular. square matrix.
Multiplying 1 to the model equation from left
11 = 1A, +1l, (5)
8
and by applying OLS to equation (5), we obtain
^
, =
_
(1A)
0
1A
_
1
(1A)
0
11
= (A
0
1
0
1A)
1
A
0
1
0
11
=
_
A
0
_
( 1
n
)
1
_
A
_
1
A
0
( 1
n
)
1
1 (since we dened 1
0
1 = ( 1
n
)
1
)
=
^
,
SUR
.
The conditional variance of error term in equation (5) is
\ ar [ 1l[ A]
. .
NJNJ
= 1
_
1l (1l)
0

= 1 [ 1ll
0
1[ A]
= 11 [ ll
0
[ A] 1
0
= 1\ ar [ l[ A] 1
0
= 1 ( 1
n
) 1
0
(since \ ar [ l[ A] = ( 1
n
) )
= 1 (1
0
1)
1
1
0
(since we dened ( 1
n
)
1
= 1
0
1 =( 1
n
) = (1
0
1)
1
)
= 11
1
(1
0
)
1
1
0
(since (1)
1
= 1
1

1
)
= 1
NJ
. (spherical errors)
Therefore, the error in equation (5) is spherical and we can apply Gauss-Markov theorem. (Remember? Gauss-
Markov theorem says LS estimator is ecient if OLS assumptions (that includes spherical errors) are satised.) Thus,
the seemingly unrelated estimator
^
,
SUR
is ecient.
2 Final Review
3
: Question 5
Consider the two equation model given by
j
1i
= r
0
1i
,
1
+n
1i
,
j
2i
= r
0
2i
,
2
+n
2i
,
where r
1i
and r
2i
are 1
1
1 and 1
2
1 vector of regressors, respectively, ,
1
and ,
2
are the corresponding unknown
vectors of parameters. Let n
i
= (n
1i
, n
2i
)
0
, and assume that
n
i
[ r
1i
, r
2i
~ i.i.d. (0,
u
) .
(1) Describe how to obtain the most ecient estimates for ,
1
and ,
2
.
Answer:
The most ecient estimator is obtained by
^
,
SUR
=
^
,
GLS
=
_
A
0
(
u
1
n
)
1
A
_
1
A
0
(
u
1
n
)
1
1,
where matrices A and 1 are constructed the same way as on lecture note (the same way as this TA note).
(2) Provide a consistent estimate for
u
.
Answer:
First, we estimate the parameter , by equation by equation OLS, and obtain
^
,
1;OLS
and
^
,
2;OLS
. Then obtain the
residuals by
^ n
1i
= j
1i
r
0
1i
^
,
1;OLS
^ n
2i
= j
2i
r
0
2i
^
,
2;OLS
.
Next, calculate the estimate of
u
by
^

u
..
22
=
1

i=1
_
^ n
1i
^ n
2i
_
_
^ n
1i
^ n
2i

p

u
(by WLLN)
3
This question is a nice practice of Kronecker product manipulation.
9
(3) Suppose that Co (n
1i
, n
2i
) = 0. How would it change the answer in (1)?
Answer:
If Co (n
1i
, n
2i
) = 0, SUR (GLS) estimator collapses to equation by equation OLS estimator (lecture note #3, p.7)
Formally, now o-diagonal elements of
u
are zero

u
=
_
o
11
0
11
0
11
o
22
_
.
Then, we dene the inverse of (conditional) variance-covariance matrix is

1
u
=
_
o
11
0
11
0
11
o
22
_
1
=
1
o
11
o
22
_
o
22
0
11
0
11
o
11
_
=
_
1
11
0
11
0
11
1
22
_
=
dene
_
o
11
0
11
0
11
o
22
_
.
and GLS weight matrix becomes
(
u
1
N
)
1
. .
2N2N
=
1
u
1
N
=
_
o
11
0
11
0
11
o
22
_
1
N
=
_
o
11
1
N
O
NN
O
NN
o
22
1
N
_
Then, SUR (GLS) estimator is
^
,
SUR
=
_
A
0
(
u
1
N
)
1
A
_
1
A
0
(
u
1
N
)
1
1
=
_
A
0
_

1
u
1
N
_
A
_
1
A
0
_

1
u
1
N
_
1
=
_
_
A
1
O
NK2
O
NK1
A
2
_
0
_
o
11
1
n
O
NN
O
NN
o
22
1
n
_ _
A
1
O
NK2
O
NK1
A
2
_
_
1 _
A
1
O
NK2
O
NK1
A
2
_
0
_
o
11
1
n
O
NN
O
NN
o
22
1
n
_ _
1
1
1
2
_
=
_
_
_
_
_
_
A
0
1
O
K1N
O
K2N
A
0
2
_
. .
transposed
_
o
11
1
n
O
NN
O
NN
o
22
1
n
_ _
A
1
O
NK2
O
NK1
A
2
_
_
_
_
_
_
1
_
A
0
1
O
K1N
O
K2N
A
0
2
_
. .
transposed
_
o
11
1
n
O
NN
O
NN
o
22
1
n
_ _
1
1
1
2
_
=
__
o
11
A
0
1
O
K1N
O
K2N
o
22
A
0
2
_ _
A
1
O
NK2
O
NK1
A
2
__
1
_
o
11
A
0
1
O
K1N
O
K2N
o
22
A
0
2
_ _
1
1
1
2
_
=
__
o
11
A
0
1
A
1
O
K1K2
O
K2K1
o
22
A
0
2
A
2
__
1
_
o
11
A
0
1
1
1
o
22
A
0
2
1
2
_
=
_
(A
0
1
A
1
)
1
A
0
1
1
1
(A
0
2
A
2
)
1
A
0
2
1
2
_
=
_
^
,
1;OLS
^
,
2;OLS
_
=
^
,
OLS
(4) Suppose that r
1i
= r
2i
for all i = 1, . . . :. How would it change the answer in (1):
Answer:
In conclusion, SUR (GLS) estimator becomes equation by equation OLS estimator.
Formally, since r
1i
= r
2i
, we have A
1
= A
2
and
A =
_
A
1
O
NK2
O
NK1
A
2
_
=
_
A
1
O
NK1
O
NK1
A
1
_
= 1
2
A
1
,
and SUR (GLS) estimator becomes (exactly the same as lecture note #3, p,6) (For Kronecker product manipulation,
10
please see the footnote
4
)
^
,
SUR
=
_
A
0
(
u
1
N
)
1
A
_
1
A
0
(
u
1
N
)
1
1
=
_
(1
2
A
1
)
0
(
u
1
N
)
1
(1
2
A
1
)
_
1
(1
2
A
1
)
0
(
u
1
N
)
1
1 (substituting A = 1
2
A
1
)
=
_
(1
2
A
0
1
) (
u
1
N
)
1
(1
2
A
1
)
_
1
(1
2
A
0
1
) (
u
1
N
)
1
1 (since (1)
0
=
0
1
0
)
=
_
(1
2
A
0
1
)
_

1
u
1
N
_
(1
2
A
1
)
_
1
(1
2
A
0
1
)
_

1
u
1
N
_
1 (since (1)
1
=
1
1
1
)
=
_

1
u
A
0
1
A
1
_
1
_

1
u
A
0
1
_
1 (since (1) (C 1) = (C 11) )
=
_

u
(A
0
1
A
1
)
1
_
_

1
u
A
0
1
_
1 =
_
1
2
(A
0
1
A
1
)
1
A
0
1
_
1
=
_
(A
0
1
A
1
)
1
A
0
1
O
k1N
O
k1N
(A
0
1
A
1
)
1
A
0
1
_
_

_
j
1
..
N1
j
2
..
N1
_

_
(substituting 1
..
2N1
=
_
j
1
j
2
_
)
=
_
(A
0
1
A
1
)
1
A
0
1
j
1
(A
0
1
A
1
)
1
A
0
1
j
1
_
=
_
(A
0
1
A
1
)
1
A
0
1
j
1
(A
0
2
A
2
)
1
A
0
2
j
2
_
=
_
^
,
1;OLS
^
,
2;OLS
_
. (since A
1
= A
2
)
3 Final 2006: Question 2
Consider the following seemingly unrelated regression (SUR) model
j
1i
= r
0
1i

1
+n
1i
,
j
2i
= r
0
2i

2
+n
2i
,
j
3i
= r
0
3i

3
+n
3i
,
where n
i
[ r ~i.i.d.(0, ) , n
i
= (n
1i
, n
2i
, n
3i
)
0
. It is given that r
1i
= r
2i
for all i = 1, . . . .
(1) Let ^
1
, ^
2
, and ^
3
be the least-square estimates for
1
,
2
, and
3
, from three separate regressions of j
1i
on r
1i
,
j
2i
on r
2i
, and j
3i
on r
3i
, respectively. Would a simultaneous LS regression of j
1i
, j
2i
and j
3i
on r
1i
, r
2i
, and r
3i
yield
the same estimates for three parameter vectors
1
,
2
and
3
? Explain.
Answer: Yes
Writing the model with matrix notation
1 = A, +l
1
..
3N1
= A
..
3N(2k1+k3)
,
..
(2k1+k3)1
+ l
..
3N1
The simultaneous LS (SLS) estimator is (here I do not plug in A
1
= A
2
and /
1
= /
2
until the last moment to explain
4
Kronecker product namipulations
(A B)
1
= A
1
B
1
(A B)
0
= A
0
B
0
(A B) (C D) = AC BD
A (B + C) = A B + A C
Do not confuse above manipulations with usual matrix manipulations. Especially notice that
(AB)
1
= B
1
A
1
(AB)
0
= B
0
A
0
:
11
the dimensionalities of matrix operations)
^
,
SLS
= (A
0
A)
1
A
0
1
=
_
_
_
_
A
1
O
Nk2
O
Nk3
O
Nk1
A
2
O
Nk3
O
Nk1
O
Nk2
A
3
_
_
0
_
_
A
1
O
Nk2
O
Nk3
O
Nk1
A
2
O
Nk3
O
Nk1
O
Nk2
A
3
_
_
_
_
1

_
_
A
1
O
Nk2
O
Nk3
O
Nk1
A
2
O
Nk3
O
Nk1
O
Nk2
A
3
_
_
0
_
_
j
1
j
2
j
3
_
_
=
_
_
_
_
_
_
_
_
_
A
0
1
O
k1N
O
k1N
O
k2N
A
0
2
O
k3N
O
k3N
O
k3N
A
0
3
_
_
. .
transposed
_
_
A
1
O
Nk2
O
Nk3
O
Nk1
A
2
O
Nk3
O
Nk1
O
Nk2
A
3
_
_
_
_
_
_
_
_
_
1

_
_
A
0
1
O
k1N
O
k1N
O
k2N
A
0
2
O
k3N
O
k3N
O
k3N
A
0
3
_
_
. .
transposed
_
_
j
1
j
2
j
3
_
_
=
_
_
_
_
A
0
1
A
1
O
k1k2
O
k1k3
O
k2k1
A
0
2
A
2
O
k2k3
O
k3k1
O
k3k2
A
0
3
A
3
_
_
_
_
1

_
_
A
0
1
j
1
A
0
2
j
2
A
0
3
j
3
_
_
=
_
_
(A
0
1
A
1
)
1
O
k1k2
O
k1k3
O
k2k1
(A
0
2
A
2
)
1
O
k2k3
O
k3k1
O
k3k2
(A
0
3
A
3
)
1
_
_

_
_
A
0
1
j
1
A
0
2
j
2
A
0
3
j
3
_
_
=
_
_
(A
0
1
A
1
)
1
A
0
1
j
1
(A
0
2
A
2
)
1
A
0
2
j
2
(A
0
3
A
3
)
1
A
0
3
j
3
_
_
=
_
_
(A
0
1
A
1
)
1
A
0
1
j
1
(A
0
1
A
1
)
1
A
0
1
j
2
(A
0
3
A
3
)
1
A
0
3
j
3
_
_
(since A
1
= A
2
)
=
_

_
^
,
1;OLS
^
,
2;OLS
^
,
3;OLS
_

_.
This is equation by equation OLS.
(2) Would a GLS regression yield the same estimates for
1
,
2
, and
3
as in (1)? Explain.
Answer: No
The GLS estimator is given by
^
,
GLS
=
^
,
SUR
=
_
A
0
( 1
N
)
1
A
_
1
A
0
( 1
N
)
1
1
=
_
A
0
_

1
1
N
_
A
_
1
A
0
_

1
1
N
_
1
,= (A
0
A)
1
A
0
1 =
^
,
OLS
See footnote
5
.
(3) Suppose now that
1
=
2
. Would it be possible to obtain the GLS estimator as in (2)?
Answer:
If
1
=
2
, we can pool the model as
_
_
j
1
j
2
j
3
_
_
=
_
_
A
1
O
Nk3
A
2
O
Nk3
O
Nk1
A
3
_
_
_

1

3
_
+
_
_
l
1
l
2
l
3
_
_
.
By applying the same procedure as (2), we obtain GLS (SUR) estimator.
(4) Describe in detail how to obtain the feasible GLS estimator for
1
,
2
, and
3
.
Answer:
Feasible GLS is obtained by three steps.
Step 1:
Estimating the model by equation by equation OLS, and obtain
^
,
1;OLS
,
^
,
2;OLS
, and
^
,
3;OLS
Then, obtain the residuals by
^ n
1i
= j
1i
r
0
1i
^
,
1;OLS
^ n
2i
= j
2i
r
0
2i
^
,
2;OLS
^ n
3i
= j
3i
r
0
3i
^
,
3;OLS
5
If we further assume x
i1
= x
i2
= x
i3
in this question, OLS and GLS (SUR) estimators are the same (as on lecture note #3, p.6)
12
Step 2:
Calculate the estimate of
u
by
^

..
33
=
1

i=1
_
_
^ n
1i
^ n
2i
^ n
3i
_
_
_
^ n
1i
^ n
2i
^ n
3i

.
Step 3:
Applying
^

u
to SUR (GLS) estimator
^
,
FGLS
=
_
A
0
_
^

u
1
n
_
1
A
_
1
A
0
_
^

u
1
n
_
1
1
(5) Show that the feasible GLS estimator in (4) Can be viewed as a GMM estimator.
Answer:
We will discuss this question when we study GMM.
4 Comp 2003F Part III (Buchinsky): Question 1
Consider the two-equation linear model given by
j
1i
= r
0
1i
,
1
+n
1i
,
j
2i
= r
0
2i
,
2
+n
2i
,
where r
1i
and r
2i
are /
1
1 and /
2
1 vector of regressors, respectively, ,
1
and ,
2
are the corresponding unknown
vector of parameters. Assume /
2
/
1
. Let n
i
= (n
1i
, n
2i
)
0
, and assume that
n
i
[ r
1i
, r
2i
~ i.i.d. (0,
u
) .
(1) Describe how to obtain the most ecient estimates for ,
1
and ,
2
.
Answer:
The most ecient estimator is obtained by GLS method
^
,
GLS
=
_
_
A
0
(
u
1
N
)
1
A
_
1
A
0
(
u
1
N
)
1
1
_
Showing GLS estimator is actually most ecient. Decomposing the inverse of weight matrix into the factor 1s
(
u
1
N
)
1
= 1
0
1.
Then, multiplying 1 to the model equation from left
1 = A, +l
we obtain
11 = 1A, +1l (6)
Then, calculating the conditional variance of error term in equation (6)
\ ar [1l] = 1\ ar [l] 1
0
= 1 (
u
1
N
) 1
0
= 1
2N
,
because
(
u
1
N
)
1
= 1
0
1
= (1
0
)
1
(
u
1
N
)
1
= 1 (multiplying (1
0
)
1
from left)
= (1
0
)
1
(
u
1
N
)
1
1
1
= 1
2N
((multiplying (1
0
)
1
from right)
= 1 (
u
1
N
)
1
1
0
= 1
2N
(take inverse, note (1C)
1
= C
1
1
1

1
).
Therefore, the equation (6) has a spherical errors and we can apply Gauss-Markov theorem (Gauss-Markov theorem:
OLS estimator is BLUE under classical linear regressions).
(2) Provide consistent estimate for
u
.
13
Answer:
Consistent estimate for
u
is dened as follows.
First, we estimating the model by equation by equation OLS, and obtain
^
,
1;OLS
and
^
,
2;OLS
Then, obtain the residuals by
^ n
1i
= j
1i
r
0
1i
^
,
1;OLS
^ n
2i
= j
2i
r
0
2i
^
,
2;OLS
.
Next, Calculate the estimate of
u
by
^

u
..
22
=
1

i=1
_
^ n
1i
^ n
2i
_
_
^ n
1i
^ n
2i

p

u
. (by WLLN)
This estimate is consistent, since
^
,
1;OLS
and
^
,
2;OLS
are consistent estimators of ,
1
and ,
2
.
(3)
6
Suppose Co (n
1i
, n
2i
) = o
12
, where o
12
is a known parameter (i.e., it is a number that the econometrician knows).
How would it change the answer in (1)?
Answer:
Dene the inverse of
u
as

1
u
=
_
o
11
o
12
o
21
o
22
_
.
Then, we have the inverse of weight matrix

1
u
1
N
. .
2N2N
=
_
o
11
1
N
o
12
1
N
o
21
1
N
o
22
1
N
_
Now,
^
,
GLS
becomes
^
,
GLS
=
_
^
,
1;GLS
^
,
2;GLS
_
=
_
A
0
(
u
1
N
)
1
A
_
1
A
0
(
u
1
N
)
1
1 =
_
A
0
_

1
u
1
N
_
A
_
1
A
0
_

1
u
1
N
_
1
=
_
_
A
1
O
Nk2
O
Nk1
A
2
_
0
_
o
11
1
N
o
12
1
N
o
21
1
N
o
22
1
N
_ _
A
1
O
Nk2
O
Nk1
A
2
_
_
1

_
A
1
O
Nk2
O
Nk1
A
2
_
0
_
o
11
1
N
o
12
1
N
o
21
1
N
o
22
1
N
_ _
j
1
j
2
_
=
_
_
_
_
_
_
A
0
1
O
k1N
O
k2N
A
0
2
_
. .
transposed
_
o
11
1
N
o
12
1
N
o
21
1
N
o
22
1
N
_ _
A
1
O
Nk2
O
Nk1
A
2
_
_
_
_
_
_
1
_
A
0
1
O
k1N
O
k2N
A
0
2
_
. .
transposed
_
o
11
1
N
o
12
1
N
o
21
1
N
o
22
1
N
_ _
j
1
j
2
_
=
__
o
11
A
0
1
o
12
A
0
1
o
21
A
0
2
o
22
A
0
2
_ _
A
1
O
Nk2
O
Nk1
A
2
__
1
_
o
11
A
0
1
o
12
A
0
1
o
21
A
0
2
o
22
A
0
2
_ _
j
1
j
2
_
=
_
o
11
A
0
1
A o
12
A
0
1
A
2
o
21
A
0
2
A
1
o
22
A
0
2
A
2
_
1
_
o
11
A
0
1
j
1
+o
12
A
0
1
j
2
o
21
A
0
2
j
1
+o
22
A
0
2
j
2
_
.
Arranging the above equation.
_
o
11
A
0
1
A o
12
A
0
1
A
2
o
21
A
0
2
A
1
o
22
A
0
2
A
2
_
_
^
,
1;GLS
^
,
2;GLS
_
=
_
o
11
A
0
1
j
1
+o
12
A
0
1
j
2
o
21
A
0
2
j
1
+o
22
A
0
2
j
2
_
Now, aiming to solve the above equation for
^
,
1;GLS
and
^
,
2;GLS
,
6
Question (3) and (4) are time consuming
14
_
o
11
A
0
1
A
1
^
,
1;GLS
+o
12
A
0
1
A
2
^
,
2;GLS
= o
11
A
0
1
j
1
+o
12
A
0
1
j
2
o
21
A
0
2
A
1
^
,
1;GLS
+o
22
A
0
2
A
2
^
,
2;GLS
= o
21
A
0
2
j
1
+o
22
A
0
2
j
2
_
o
11
A
0
1
A
1
^
,
1;GLS
= o
11
A
0
1
j
1
+o
12
A
0
1
j
2
o
12
A
0
1
A
2
^
,
2;GLS
o
22
A
0
2
A
2
^
,
2;GLS
= o
21
A
0
2
j
1
+o
22
A
0
2
j
2
o
21
A
0
2
A
1
^
,
1;GLS
_
^
,
1;GLS
= (A
0
1
A
1
)
1
A
0
1
j
1
+

12

11
(A
0
1
A)
1
A
0
1
j
2

12

11
(A
0
1
A)
1
A
0
1
A
2
^
,
2;GLS
^
,
2;GLS
=

21

22
(A
0
2
A
2
)
1
A
0
2
j
1
+ (A
0
2
A
2
)
1
A
0
2
j
2

21

22
(A
0
2
A
2
)
1
A
0
2
A
1
^
,
1;GLS
_

_
^
,
1;GLS
= (A
0
1
A
1
)
1
A
0
1
j
1
. .
=
^

1;OLS
+

12

11
(A
0
1
A)
1
A
0
1
. .
A1
_
j
2
A
2
^
,
2;GLS
_
^
,
2;GLS
= (A
0
2
A
2
)
1
A
0
2
j
2
. .
=
^

2;OLS
+

21

22
(A
0
2
A
2
)
1
A
0
2
. .
A2
_
j
1
A
1
^
,
1;GLS
_
and we obtain system equations
_
_
_
^
,
1;GLS
=
^
,
1;OLS
+

12

11

1
_
j
2
A
2
^
,
2;GLS
_
^
,
2;GLS
=
^
,
2;OLS
+

21

22

2
_
j
1
A
1
^
,
1;GLS
_
(7)
where we dened

1
= (A
0
1
A)
1
A
0
1

2
= (A
0
2
A
2
)
1
A
0
2
.
Now, we assume

u
=
_
o
11
o
12
o
21
o
22
_
.
Then, we have the relation

1
u
=
1
o
11
o
22
o
12
o
21
_
o
22
o
12
o
21
o
11
_
=
_
22
11221221

12
11221221

21
11221221
11
11221221
_
=
we dened before
_
o
11
o
12
o
21
o
22
_
.
We know o
12
= o
21
(given in the setting of this question). Also, we can consistently estimate o
11
and o
22
by using
equation by equation OLS estimates
^
,
1;OLS
and
^
,
2;OLS
^ o
11
=
1

i=1
j
1i
r
0
1i
^
,
1;OLS
p
o
11
^ o
22
=
1

i=1
j
2i
r
0
1i
^
,
2;OLS
p
o
22
.
Then, by substituting ^ o
11
, ^ o
22
, o
12
, o
21
, we can get
_
^ o
11
^ o
12
^ o
21
^ o
22
_
. .
super scripts
=
_
^ 22
^ 11^ 221221

12
^ 11^ 221221

21
^ 11^ 221221
^ 11
^ 11^ 221221
_
. .
sub scripts
Therefore, we can solve the above system equations (7) for
^
,
1;GLS
and
^
,
2;GLS
(by plug in j
1
, j
2
, A
1
, A
2
, ^ o
11
, ^ o
22
, ^ o
12
, ^ o
21
).
(4)
7
Suppose that all the variable in r
1i
are also in r
2i
, that is, r
1i
r
2i
for all i = 1, . . . :. How would it change the
answer in (1) with respect to the parameter vector ,
1
and with respect to the parameter ,
2
?
Answer:
We re-construct model with condition r
1i
r
2i
. Assume
r
2i
=
_
r
1i
n
2i
_
and ,
2
=
_

2
c
2
_
.
7
This question is calculation intensive. If you see this type of question in exam, its better to solve other easy questions rst.
15
Then, the model becomes
j
1i
= r
0
1i
,
1
+n
1i
j
2i
= r
0
1i

2
+n
2i
c
2
+n
2i
Matrix Notation is
j
1
= A
1
,
1
+l
j
2
= A
2
,
2
+l = [A
1
, \
2
]
_

2
c
2
_
+l
For future calculation, we derive (believe me, we will use these products soon)

1
1
X2
= (A
0
1
A
1
)
1
A
0
1
1
X2
= (A
0
1
A
1
)
1
(1
X2
A
1
)
0
= (A
0
1
A
1
)
1
A
0
1
(since 1
X2
A
1
= A
1
)
=
1
(simplied)
Here, the projection of A
1
on A
2
is just A
1
, because A
2
contains A
1
(A
1
is included in the hyper plane space of A
2
).
Also,

2
1
X1
= (A
0
2
A
2
)
1
A
0
2
1
X1
= (A
0
2
A
2
)
1
[A
1
, \
2
]
0
1
X1
= (A
0
2
A
2
)
1
_
A
0
\
0
2
_
1
X1
= (A
0
2
A
2
)
1
_
A
0
1
1
X1
\
0
2
1
X1
_
= (A
0
2
A
2
)
1
_
A
0
1
A
1
(A
1
A
1
)
1
A
0
1
\
0
2
1
X1
_
=
_
(A
0
2
A
2
)
1
A
0
1
(A
0
2
A
2
)
1
\
0
2
1
X1
_
(not simplied)
Now, by arranging the system equations
_
_
_
^
,
1;GLS
=
^
,
1;OLS
+

12

11

1
_
j
2
A
2
^
,
2;GLS
_
^
,
2;GLS
=
^
,
2;OLS
+

21

22

2
_
j
1
A
1
^
,
1;GLS
_
and
_
^
,
1;GLS
=
^
,
1;OLS
+

12

11

1
j
2

12

11

1
A
2
^
,
2;GLS
^
,
2;GLS
=
^
,
2;OLS
+

21

22

2
j
1

21

22

2
A
1
^
,
1;GLS
(8)
By substituting second equation of system (8) to rst equation
16
^
,
1;GLS
=
^
,
1;OLS
+
o
12
o
11

1
j
2

o
12
o
11

1
A
2
_
^
,
2;OLS
+
o
21
o
22

2
j
1

o
21
o
22

2
A
1
^
,
1;GLS
_
_
1
k1

o
12
o
21
o
11
o
22

1
A
2

2
A
1
_
^
,
1;GLS
=
^
,
1;OLS
+
o
12
o
11

1
j
2

o
12
o
11

1
A
2
^
,
2;OLS

o
12
o
21
o
11
o
22

1
A
2

2
j
1
_
_
_
_
1
k1

o
12
o
21
o
11
o
22

1
A
2

2
..
=(X
0
2
X)
1
X2
A
1
_
_
_
_
^
,
1;GLS
=
^
,
1;OLS
+
o
12
o
11

1
j
2

o
12
o
11

1
A
2
^
,
2;OLS
. .
=(X
0
2
X2)
1
X
0
2
y2

o
12
o
21
o
11
o
22

1
A
2

2
..
=(X
0
2
X)
1
X2
j
1
_
_
_1
k1

o
12
o
21
o
11
o
22

1
A
2
(A
0
2
A
2
) A
0
2
. .
=P
X
2
A
1
_
_
_
^
,
1;GLS
=
^
,
1;OLS
+
o
12
o
11

1
j
2

o
12
o
11

1
A
2
(A
0
2
A
2
) A
0
2
. .
=P
X
2
j
2

o
12
o
21
o
11
o
22

1
A
2
(A
0
2
A
2
)
1
A
0
2
. .
=P
X
2
j
1
_
_
_1
k1

o
12
o
21
o
11
o
22

1
1
X2
. .
=A1
A
1
_
_
_
^
,
1;GLS
=
^
,
1;OLS
+
o
12
o
11

1
j
2

o
12
o
11

1
1
X2
. .
j
2
=A1

o
12
o
21
o
11
o
22

1
1
X2
. .
=A1
j
1
_
1
o
12
o
21
o
11
o
22

1
A
1
_
^
,
1;GLS
=
^
,
1;OLS
+
o
12
o
11

1
j
2
. .
cancel out

o
12
o
11

1
j
2
. .
cancel out

o
12
o
21
o
11
o
22

1
j
1
_
1
k1

o
12
o
21
o
11
o
22
(A
0
1
A
1
)
1
A
0
1
A
1
_
^
,
1;GLS
=
^
,
1;OLS

o
12
o
21
o
11
o
22
^
,
1;OLS
_
1
k1

o
12
o
21
o
11
o
22
1
k1
_
. .
cancel out
^
,
1;GLS
=
_
1
k1

o
12
o
21
o
11
o
22
1
k1
_
. .
cancel out
^
,
1;OLS
^
,
1;GLS
=
^
,
1;OLS
So for the estimation of ,
1
, GLS estimator is equivalent to OLS estimator. Intuition here is that since A
1
is included
in A
2
,
^
,
1;GLS
cannot exploit the information of o-diagonal elements of variance-covariance matrix.
Next, by substituting rst equation of system (8) to second equation
^
,
2;GLS
=
^
,
2;OLS
+
o
21
o
22

2
j
1

o
21
o
22

2
A
1
_
^
,
1;OLS
+
o
12
o
11

1
j
2

o
12
o
11

1
A
2
^
,
2;GLS
_
_
_
_
_
1
k2

o
21
o
12
o
22
o
11

2
A
1

1
..
=(X
0
1
X1)
1
X1
A
2
_
_
_
_
^
,
2;GLS
=
^
,
2;OLS
+
o
21
o
22

2
j
1

o
21
o
22

2
A
1
^
,
1;OLS
. .
=(X
0
1
X1)
1
X1y1

o
21
o
12
o
22
o
11

2
A
1

1
..
=(X
0
1
X1)
1
X1
j
2
_
_
_1
k2

o
21
o
12
o
22
o
11

2
A
1
(A
0
1
A)
1
A
0
1
. .
=P
X
1
A
2
_
_
_
^
,
2;GLS
=
^
,
2;OLS
+
o
21
o
22

2
j
1

o
21
o
22

2
A
1
(A
0
1
A)
1
A
0
1
. .
=P
X
1
j
1

o
21
o
12
o
22
o
11

2
A
1
(A
0
1
A)
1
A
0
1
. .
=P
X
1
j
2
_
1
k2

o
21
o
12
o
22
o
11

2
1
X1
A
2
_
^
,
2;GLS
=
^
,
2;OLS
+
o
21
o
22

2
j
1
. .
not cancel out

o
21
o
22

2
1
X1
j
1
. .
not cancel out

o
21
o
12
o
22
o
11

2
1
X1
j
2
^
,
2;GLS
=
_
1
k2
o
21
o
12
o
22
o
11

2
1
X1
A
2
_
1

_
^
,
2;OLS
+
o
21
o
22

2
j
1

o
21
o
22

2
1
X1
j
1

o
21
o
12
o
22
o
11

2
1
X1
j
2
_
Thus,
^
,
2;GLS
is not equivalent to
^
,
2;OLS
. Again, intuition is that since A
2
has some elements that A
1
does not have,
^
,
2;GLS
can exploit the information of o-diagonal elements of variance-covariance matrix.
17
(5) Suppose now that
j
1i
= r
0
1i
,
1
+j
2i
+n
1i
,
j
2i
= r
0
2i
,
2
+n
2i
.
Provide a method for consistently estimating ,
1
and ,
2
.
Answer:
This is the question of "endogenous simultaneous equation system" that we do not cover this year. So, let me
compromise this question.
5 Comp 2006F Part III (Buchinsky): Question 1
Consider the Seemingly Unrelated Regression (SUR) model
j
ji
= r
0
ji
,
j
+n
ji
(i = 1, . . . , : , = 1, . . . , J)
(1) State the condition for n
ji
that would allow one to estimate parameter vector ,
j
, , = 1, . . . , J from J separate
least-squares (LS) regression.
Answer:
Writing the model in matrix notation
1 = A, +l,
where vectors and matrices A, 1, l and , are constructed as lecture note (as this TA note).
The LS regression provides us the estimator
^
,
LS
= (A
0
A)
1
A
0
1.
The assumption of strict exogeneity
1 [ n
ji
[ all r
ji
for , = 1, , J and i = 1, , ] = 0
11
or equivalently,
1 [ l[ A] = O
NJ
guarantees the constancy of
^
,
LS
.
(2) Suppose that J = 3. Also, assume that r
1i
and r
2i
are vector of exogenous regressors, while for r
3i
we have
1 [n
3i
r
3i
] ,= 0. State the condition(s) under which one will be able to obtain consistent estimator for ,
3
. Be as precise as
possible in stating your assumption(s).
Answer:
Note that we have the matrix notation for the third equation
j
3
= A
3
,
3
+l
3
.
,
3
can be identied if the instrument .
i
(| 1)-vector is available. Assume conditions
1 [.
i
n
3i
] = 0
l1
(uncorrelated to error)
1 [.
i
r
0
3i
] =
zx
,= O
ldim(x3i)
. (.
i
and r
i
are correlated)
Then, stack up instrument as
7
..
Nl
=
_

_
.
0
1
.
.
.
.
0
N
_

_.
Multiplying 1
Z
= 7
_
7
0
7
_
1
7
0
to the matrix notation
1
Z
j
3
= 1
Z
A
3
,
3
+1
Z
l
3
,
and applying OLS to the above equation, we get 2SLS estimator
^
,
3;2SLS
=
_
(1
Z
A
3
)
0
1
Z
A
3
_
(1
Z
A
3
)
0
1
Z
j
3
= (A
0
3
1
Z
A
3
)
1
A
0
3
1
Z
j
3
.
18
Under conditions above,
^
,
3;2SLS
is consistent because
^
,
3;2SLS
= ,
3
+ (A
0
3
1
Z
A
3
)
1
A
0
3
1
Z
l
= ,
3
+ (A
0
3
1
Z
A
3
)
1
A
0
3
7
_
7
0
7
_
1
7
0
l
= ,
3
+ (A
0
3
1
Z
A
3
)
1
A
0
3
7
_
1

7
0
7
_
1
1

7
0
l
. .
p
!0
l1
p
,
3
.
(3) Suppose now that n
i
= (n
1i
, . . . , n
Ji
) ; we have
n
i
[ r
1i
, r
2i
, r
3i
~ i.i.d. (0, ) .
How would you obtain a feasible generalized least squares (FGLS) estimator for ,
1
, . . . , ,
J
. Justify your answer at
each and every stage.
Answer:
The feasible GLS estimator obtained by three steps.
Step 1:
Implement equation by equation OLS and obtain
^
,
1;OLS
= (A
0
1
A
1
)
1
A
1
j
1
.
.
.
^
,
j;OLS
=
_
A
0
j
A
j
_
1
A
j
j
j
.
.
.
^
,
j;OLS
=
_
A
0
j
A
j
_
1
A
j
j
j
and obtain residual vector
_

_
^ n
1i
.
.
.
^ n
ji
.
.
.
^ n
Ji
_

_
. .
J1
=
_

_
j
1i
r
0
1i
^
,
1;OLS
.
.
.
j
ji
r
0
ji
^
,
j;OLS
.
.
.
j
Ji
r
0
Ji
^
,
J;OLS
_

_
.
Step 2:
Calculate the estimate of
^

..
JJ
=
1

i=1
_

_
^ n
1i
.
.
.
^ n
ji
.
.
.
^ n
Ji
_

_
. .
J1
_
^ n
1i
^ n
ji
^ n
Ji

. .
1J
.
Step 3:
Feasible GLS estimator is obtained by
^
,
FGLS
=
_
A
0
_
^
1
N
_
1
A
_
1
A
0
_
^
1
N
_
1
1.
19