1

Econometrics 1
Lecture 1
Classical Linear Regression Analysis
2
Application of mathematical statistics to economic data
to send empirical support:
a. Economic theory postulates a qualitative relation
b. Mathematical economics turns economic theory
in equations
c. Economic statistics concerns with collecting,
processing and presenting economic data
d. Econometricians estimate precise numerical
estimates of these relations
What is Econometrics ?

3
Econometrics
Theoretical Applied
Classical Bayesian Classical Bayesian
Branches of Econometrics

4
Traditional or Classical
Methodology of
Econometrics
- Statement of hypothesis
- Specification of the
mathematical model
- Specification of
econometric model
- Data collection
- Estimation of
parameters of the
econometric model
- Hypothesis testing
- Forecasting or
prediction
- Using model for control
or policy analysis
Mehtodology of Bayesian
Econometrics
- Bayesian prior
- Sample information
- Posterior information
Econometric Methodology
5
Assume a Simple Linear regression model:
i
e
i
x
i
Y + + =
2 1
| |
Main assumptions about the error term
i
e are following:
- Mean of
i
e is zero for every observations of
i
x , 0 =
(

i
e E
- variance of
i
e
is constant
2
var o =
(

i
e
for every ith observation
-
j i all for
j
e
i
e = =0 ) cov(
; this also means there is no autocorrelation or
heteroscedasticity; errors are homoscedatic and independent of each other
- there is no correlation between
i
e
and the explanatory variable
i
x
; 0 =
(

i
x
i
e E
- explanatory variable,
i
x
, is exogenous, not random
- variance of the dependent variable is equal to the variance of the error
term
2
var var o = =
(

|
.
|

\
|
i
e
i
y
6
- i
x
i
y
2

| | + =
.
Y



X
Graphical Illustration of a Simple Linear Regression Model
Each dot in the above graph represents an observation. Some
observations lie above the least square
i
Y

line and other observations lie

below it.
These errors represent all sorts elements missing from this relationship.
Some of them might be due to the missing variables, others might be due
to measurement errors, still other may be from the mis-specification of
the relationship.
The least square line is the line best fits the data set. Differences between
each observation and the
i
Y

line is represented by error terms

i
e . As
some of them are above the line and others below the line, positive errors
cancel out with the negative errors. Note that the least square line passes
through the average values of variables X and Y;
X
, Y .
7
Minimisation of Error Sum Square
Errors are given by
i
x
i
Y
i
e
2 1
| | = . Some of them
are positive and some others are negative. Since
mean of these errors is zero, 0 =
(

i
e E , it is customary
to take sum squared errors and estimate the unknown
parameters
1
|
and
2
|
that minimise the sum squared
errors.

=
|
.
|

\
|
i
i
x
i
Y
i
i
e S
2
2 1
2
| | (1)
Sign of each and every squared error would be
positive
0
2
>
|
|
|
.
|

\
|

i
e
when |
|
.
|

\
|
2
, 0 ~ o N
i
e
.
2
~
2
n
i
i
e S _

=
distribution where subscript n stands for
degrees of freedom which equals the number of terms
in S
Normal equations of the least square estimator are
obtained by minimising S function (1) with respect to
1
|
and 2
|
.
8
Derivation of Normal Equations
( ) 0 1
2 1
2
1
=

=
c
c
|
.
|

\
|
i
x
i
Y
S
| |
|
and
0
2 1
2
2
=

=
c
c
|
.
|

\
|
|
.
|

\
|
i
x
i
x
i
Y
S
| |
|
Thus normal equations are

+ =

i
i
x N
i
i
y
2 1
| |
(2)

i
i
x
i
i
x
i
i
y
i
x
2
2 1
| | (3)
This is a system of two equations, (2) and (3) , and
two unknowns
1
|
and
2
|
. All other values such as

t
x
,
t
y
t
x
,
2
t
x
,
t
y
and N are known from the
sample information on X and Y. In order get value of
2
|
eliminate
1
|
by multiplying the (2) by

t
x
and
(3) by N and take a difference of the resulting two
equations.
9
OLS Estimators
2
2 1 |
|
|
.
|

\
|

=

i
i
x
i
i
x N
i
i
y
i
i
x | | (4)

i
i
x N
i
i
x N
i
i
y
i
x N
2
2 1
| |
(5)
Now subtracting (5) from (4) we get the estimator for
2
|
.
2
2
2

|
|
|
.
|

\
|

=
i
i
x
i
i
x N
i i i
i
y
i
x
i
y
i
x N
|
(6)
Estimator for
1
|
can be found by dividing both sides
of (2) by N and using the average values
x
and
y
.
x y
2

| | =
(7)
10
Food expenditure and income: data and prediction
Y X Xy xsquare ysquare Ypred Sqpredy prede sqprede
4 5 20 25 16 2.866285 8.21559 1.133715 1.28531
6 8 48 64 36 5.742472 32.97598 0.257528 0.066321
7 10 70 100 49 7.65993 58.67453 -0.65993 0.435508
8 12 96 144 64 9.577388 91.72636 -1.57739 2.488153
11 14 154 196 121 11.49485 132.1315 -0.49485 0.244873
15 17 255 289 225 14.37103 206.5266 0.628967 0.395599
18 20 360 400 324 17.24722 297.4666 0.75278 0.566678
22 25 550 625 484 22.04087 485.7997 -0.04087 0.00167
Sumy Sumx Sumxy sumxsq sumysq 36.4218 Smsqpred
y
smsqpred
e
91 111 1553 1843 1319 127.4218 1313.517 -3.9E-05 5.484111
An Example of OLS Estimation
11
2
2
2

|
.
|

\
|

=


i
i
i
i
i i i
i i i i
x x N
y x y x N
|
Or using the values from the above table.
95873 . 0
2423
2323
12321 14744
10101 12424
) 111 ( ) 1843 ( 8
) 91 ( 111 ) 1553 ( 8

2
2
= =

= |
(8)
92724 . 1 30224 . 13 375 . 11 ) 875 . 13 ( 95872 . 0 375 . 11
8
111
95872 . 0
8
91

2 1
= = = = = x y | |
(9)
The fitted regression line is
i i i
x x y 95873 . 0 92724 . 1

2 1
+ = + = | |
(10)
Estimates
12
Both slops and intercepts make economic sense. In this sample expenditure
on foods is determined by weekly income of an individual, people spend
95.6% percent of their weekly income in food expenditure. People who do
not have any income receive a income subsidy of 1.93 pence per week.

- Mean prediction
We can use equation (10) to find the predicted values
i
Y

for each
observation on
i
x . These are reported as YPRED in the above table. If the
weekly income is 40 predicted food expenditure will be 36.422. Error terms
are also estimated using the fact that
i
x
i
y
i
x
i
y
i
y
i
y
i
e 95873 . 0 92724 . 1
2

= = = | |
These predicted errors are reported as prede in the above table. Note that as
expected some of the errors are negative and some other are positive.
Interpretation and Prediction
13
Prediction of food expenditure
0.00
5.00
10.00
15.00
20.00
25.00
0 1 2 3 4 5 6 7 8 9
Income
P
r
e
d
i
c
t
e
d

f
o
o
d

e
x
p
e
n
d
i
t
u
r
e
Ypred
Prediction of Food Expenditure
14
The definition of elasticity of food expenditure on income is given by
1683 . 1
375 . 11
875 . 13
95783 . 0 95783 . 0 = = =
c
c
=
c
c
=
Y
X
Y
X
X
Y
X X
Y Y
q
This suggests that the expenditure on food is elastic around the mean.
There will be 17 pence more expenditure to every 1 rise in weekly
income.
Use of regression estimates to calculate the
elasticities

15
0. Create a Metrics directory in G: drive.
1. Login to the network
2. At start choose Applications\Economics\Professional Shazam
Limdep also is available there if you are familiar with it.
3. You have an editor in the Shazam program to write your program. An
econometric program involves following four steps while compiling and
computing the model.
1. declaring the sample size
2. reading the data for each variables declared
3. calculations (checking the discriptive statistics if the mean and
variance, correlation ; make sure that there are no missing
observations0
4. Using the standard Shazam routines for estimation,
such as OLS x Y; Arima x
5. Interpreting the results whether they make sense according to the
economic theory.
4. Click on Shazam. Now you should be in the Shazam program.
Click on File/New, it will bring you to the Shazam editor.
6. Write a Shazam program similar to the one as given in the example
below. Save this Shazam file in your own directory in G:\metrics directory
Hints to get into the Shazam program in the
Network

16
How to Read Data in Shazam?

For small data files cut your data and past
in the Shazam editor.
For large data file you can read the data
directly from the file. The data file should
be in the text format. If your data is in
Excell save your data in the text format
using save as option or make a number
small files of data and combine those data
using Shazam program.
There are more examples in File/Open/Intro
option in the menu. There is also a demo
which can bring you various features of
Shazam. It is worth trying if this is your
first encounter with Shazam.
17
7. When you have your program written,
then click on Run, and Run Batch to
execute your program.
8. If everything is alright you will get
Shazam working behind the screen and
displaying output in the screen. You may
save your result file in your directory if
you wish by using copy and past in the
edit menu.
9. Get more practice on several aspects of
the programming with Shazam such as reading
a data file, transforming variables by
taking log or lag or square, plotting one
variable against another one using plot x
y /gnu line only, saving a variable to use
later on. Also write a couple of lines to
read regression estimates and diagnostics.
10. Consult the Short Loan Section in the
Library to borrow a hard copy of the Shazam
manual. You can always use the online
manual inside the Shazam program which you
can get by clicking at Help and Visit
Shazam Online option while you are in the
Shazam programme.
Getting Around with Shazam
18
A Simple Example of Shazam
sample 1-10
read y x1 x2
3.5 15 16
4.5 20 13
5 30 10
6 42 7
7 50 7
9 54 5
8 65 4
10 72 3
12 85 3.5
14 90 2
ols y x1 x2 /cov=b anova
CONFID X1 X2
confid x1 x2 / TCRIT=3.499
gen1 srb =sqrt(b:1)
print b srb
ols y x1 x2 /predict=py
diagnos / het
diagnos / acf
dim p 10 2
gls y x1 x2 /omega=b
print py
stop
19
(
(
(
(
(
(
(
(
(
(
(

=
25 1
20 1
17 1
14 1
12 1
10 1
8 1
5 1
25 20 17 14 12 10 8 5
1 1 1 1 1 1 1 1
' X X =>
(

=
1843 111
111 8
' X X
(
(
(
(
(
(
(
(
(
(
(

=
22
18
15
11
8
7
6
4
25 20 17 14 12 10 8 5
1 1 1 1 1 1 1 1
'Y X => (

=
1553
91
'Y X
Simple Regression in matrix notation

20
The estimators in terms of matrix notation:
(
(
(

(
(
(

=
(

i
i i
i
i
i
i
i
i
i
i
y x
y
x x
x N
1
2
2
1

|
|
| =
( ) Y X X X ' '
1
=
(

1553
91
1843 111
111 8
1
XX XY XX XY
The desired inverse matrix is
( ) ( )
(

= =

8 111
111 1843
2423
1
'
'
1
'
1
X X Adj
X X
X X
(

+

=
(

=
(

=
(

1553 ) 8 ( ) 91 ( 111
) 1553 ( 111 ) 91 ( 1843
2423
1
1553
91
8 111
111 1843
2423
1
1553
91
1843 111
111 8

1
2
1
|
|
(

=
(

+

=
(

95873 . 0
92736 . 1
12424 10101
172383 167723
2423
1

2
1
|
|
The estimators in terms of matrix notation