Escolar Documentos
Profissional Documentos
Cultura Documentos
Roberto Casarin
University Ca Foscari, Venice
Summer School of Bayesian Econometrics
Perugia 2013
September 9, 2013
Contents
1 A Matlab Primer
1.1 Programming Languages . . . . . . . . . . .
1.2 Fourth Generation Languages (4GPL) . . .
1.3 Matlab . . . . . . . . . . . . . . . . . . . . .
1.3.1 Operators . . . . . . . . . . . . . . .
1.3.2 Logical Operators . . . . . . . . . . .
1.3.3 Creating Matrices . . . . . . . . . . .
1.3.4 Matrix Description . . . . . . . . . .
1.3.5 Other Functions . . . . . . . . . . . .
1.3.6 Loops and If Statements . . . . . . .
1.3.7 Procedures . . . . . . . . . . . . . . .
1.4 Examples . . . . . . . . . . . . . . . . . . .
1.4.1 Input, Output and Graphics . . . . .
1.4.2 Ordinary Least Square . . . . . . . .
1.4.3 A Bayesian Linear Regression Model
1.5 From Matlab to Scilab and R . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
. 1
. 5
. 6
. 6
. 6
. 7
. 7
. 7
. 8
. 9
. 9
. 9
. 11
. 12
. 17
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
21
21
22
24
25
27
3 Importance Sampling
31
3.1 Importance Sampling . . . . . . . . . . . . . . . . . . . . . . . 31
3.2 Properties of the IS Estimators . . . . . . . . . . . . . . . . . 32
3.3 Generating Student-t Variables . . . . . . . . . . . . . . . . . 34
i
ii
CONTENTS
Chapter 1
A Matlab Primer
Aim
Learn some basic facts in Matlab programming
Contents
1. Programming Languages
2. Fourth Generation Languages (4GPL)
3. Matlab
4. Examples
5. From Matlab to Scilab
1.1
Programming Languages
GAUSS: is a programming language designed to operate with and on matrices. It is a general purpose tool. As such, it is a long way from more
specialised econometric packages. On a spectrum which runs from the computer language C at one end to, say, the menu-driven econometric program
EViews at the other, GAUSS is very much at the programming end.
GRETL: Is a cross-platform software package for econometric analysis, written in the C programming language. It is free, open-source software.
HLM
ICRFS-Plus, ILOG, IDAMS, IMSL, INSTAT, ITSM
J, JMP, JMulti, JStatCom, JWAVE
KNITRO
MacAnova, Maple, Mendeley, MARS, Mathcad, Mathematica, MathPlayer, MathML, MathType, MATLAB, Matrixer, M@ximize, MetrixND,
MHTS, Microfit, MiKTeX, Minitab, MINOS, MIXOR, MLE, MLwiN,
Modeleasy, ModelQED, Modler, MOSEK, Mplus, Modula, MuPAD,
Mx.
MATLAB: It is a high-level language and more specifically a 4GPL (such as
SAS, SPSS, Stata, GAUSS) which allows matrix manipulations for numerical computing.
Octave: a high-level language, primarily intended for numerical computations. It provides a convenient command line interface for solving linear and
nonlinear problems numerically, and for performing other numerical experiments using a language that is mostly compatible with Matlab. It may also
S+, SAS, SCA, Scilab, SciPy, SciViews, Sciword, SCP, Shazam, Sigmaplot,
SIMSTAT, SOLAS, SOL, Soritec, SpaceStat, SQlite, SPAD, Speakeasy,
IBM SPSS, SsfPack, STAMP, Stata, StatCrunch, Statgraphics, Statistica, Stat/Transfer, StatsDirect, STL, Statview, SUDAAN, SVAR,
SYSTAT
SAS: is a 4GPL which allows to define a sequence of operations (statistical
analysis and data management) to be performed on data
Scilab: is 4GPL free and open source for numerical computation, similar to
Matlab
Web Decomp, WebStat, WEKA, WinIDAMS, WINKS, Windows KWIKSTAT, XploRe, Winsolve, X-12-ARIMA, XLisp-Stat, Xtremes, X(G)PL
1.2
manipulation and in this sense they are more useful for economists than the
3GPL programming languages (such as C or Fortran), where the basic data
units are all scalars. At the same time they are very flexible and allows more
expert users to use interface to procedures written in other languages such
as C, C++, or Fortran.
An important feature of Scilab and R is that the source code of their
libraries are available, which is not generally the case for Matlab and Gauss.
Finally note that Matlab, Gauss and R have a lot of proprietary and contributed libraries oriented to statistics and econometrics.
1.3
1.3.1
Matlab
Operators
Matrix Operators: + - * \ %
Element-by-element operators: .+ .- .* .\
Concatenating operators:
1.3.2
Logical Operators
1.3. MATLAB
1.3.3
Creating Matrices
let x = [ 1 2 3, 4 5 6, 7 8 9 ];
disp(x);
x = eye( 3 );
x = ones( 2, 5 );
x = ( start : inc : n );
x = rand( 4, 10 );
x = randn( 4, 10 );
y = x[ 2, : ];
1.3.4
Matrix Description
y = size( x , 1 );
y = size( x , 2 );
y = max( x, [] , 1 );
y = max( x, [] , 2 );
y = [ma,maind]=max( x );
y= mean( x );
y= prod( x );
y= cumsum( x );
y= cumprod( x );
1.3.5
Other Functions
y= diag( x );
y= inv( x );
y= log( x );
y= exp( x );
y =round( x );
y= ceil( x );
y= floor( x );
y= reshape( x,r,c );
Kronecker product: kron( x , y )
y= trimr( x,t,b );
1.3.6
for i=start:step:increment;
...
end;
if logical expression 1;
...
elseif logical expression 2;
...
else;
...
end;
Example of do loop with counter:
i=1;
while (i=100);
...
i=i+1;
end;
1.4. EXAMPLES
1.3.7
Procedures
Procedures may contain so called local variables. These variables are called
local, since you can access them only in the particular procedure.
Example: Computation of log-returns for financial assets:
function [lrt]=logret(data)
% Procedure Name: logret
% Input: T * K matrix of levels
% Output: (T-1) * K matrix of ln-returns
% Utilizing the Procedure:
% returns=_logret(prices);
% prices has to be an existing T*K matrix
% returns is the created (T-1)*K matrix
% containing the ln-returns
dat1=log(data(2:size(data,1),:));
dat2=log(data(1:size(data,1)-1,:));
lrt=dat1-dat2;
end
1.4
1.4.1
Examples
Input, Output and Graphics
%*************************************************
% basic in I/O, graphical, statistical procedures
%*************************************************
%
Load UK/EU exchange rate data
clc;
clear all;
yy= load(C:/Dottorato/Teaching/SummerSchoolBertinoro/...
TutorialAntonietta/TutorialRobAnt/AllLab/MatlabCode/ChapterMatlab/pound.txt);
%*************************************************
n=size(yy,1);
% evaluate the number of rows %
xx=(1:n);
%*************************************************
%
for end
if end
% (1) Evaluate sequentially the variance
% (2) Built a dummy variable, based on the value
%
of the variance estimated recursively
%*************************************************
wn=10;
% set the value of a variable%
s=zeros(n,1);
% define a n-dim null vector %
d=zeros(n,1);
for j=(wn+1):n;
s(j,1)=var(yy((j-wn+1):j,1));
if (s(j,1)>0.45);
d(j,1)=1;
10
end;
end;
%*************************************************
%
Some Pictures...
%*************************************************
% figure(1) to have distinct graphs
figure(1);
title(Time series data);
ylabel(Data);
xlabel(Time);
plot(xx,yy);
figure(2);
title(Time-varying log-volatility);
a=plot(xx,s,color,[1 0 0]); %[red green blue] the rgb convention
axis([1 n min(s) max(s)]);
% Set tics
figure(3);
title(Dummy);
plot(xx,d,color,[1 0 0]); %[red green blue] the rgb convention
axis([1 n -0.1 1.1]); % Set tics
%*************************************************
%
All charts in one pictures...
%*************************************************
figure(4);
subplot(3,1,1);
title(Time series data);
ylabel(Data);
xlabel(Time);
plot(xx,yy);
subplot(3,1,2);
title(Time-varying log-volatility);
plot(xx,s,color,[1 0 0]); %[red green blue] the rgb convention
axis([1 n min(s) max(s)]);
% Set tics
subplot(3,1,3);
title(Dummy);
plot(xx,d,color,[1 0 0]); %[red green blue] the rgb convention
axis([1 n -0.1 1.1]); % Set tics
%*************************************************
%
histogram
%*************************************************
figure(5);
hist(yy,50);
%*************************************************
% Save the results in a ouput file
%*************************************************
fid = fopen(C:/Dottorato/Teaching/SummerSchoolBertinoro/...
TutorialAntonietta/TutorialRobAnt/AllLab/MatlabCode/ChapterMatlab/OutPound.txt, w);
fprintf(fid, %5.2f\n, yy);
fclose(fid);
1.4. EXAMPLES
11
%*************************************************
1.4.2
Check for the correct number of input argument and if the number of
rows of x is equal to the number of rows of y
if (nargin ~= 2); error(Wrong # of arguments to ols);
else
[nobs nvar] = size(x); [nobs2 ndep] = size(y);
if (nobs ~= nobs2); error(x and y must have same # obs in ols);
end;
end;
k=nvar;
Evaluate all the statistics that are usually involved in a OLS estimation
results.y = y;
results.nobs = nobs;
results.nvar = nvar;
%
xpxi = (x*x)\eye(k);
results.beta = xpxi*(x*y);
results.yhat = x*results.beta;
results.resid = y - results.yhat;
sigu = results.resid*results.resid;
results.sige = sigu/(nobs-nvar);
tmp = (results.sige)*(diag(xpxi));
results.tstat = results.beta./(sqrt(tmp));
12
ym = y - mean(y);
rsqr1 = sigu; rsqr2 = ym*ym;
results.rsqr = 1.0 - rsqr1/rsqr2; % r-squared
rsqr1 = rsqr1/(nobs-nvar);
rsqr2 = rsqr2/(nobs-1.0);
results.rbar = 1 - (rsqr1/rsqr2); % rbar-squared
ediff = results.resid(2:nobs) - results.resid(1:nobs-1);
results.dw = (ediff*ediff)/sigu; % durbin-watson
end;
We save as a function the ols.m code and run the following simulation
example
nob=100;
x1=ones(nob,1);
x2=randn(nob,1).*((1:nob)/10);
x=[x1 x2];
sig=2;
y=x*[10; 0.9]+sig*randn(nob,1);
res=ols(y,x);
res.beta
%%
figure(1)
plot([res.yhat y]);
figure(2)
plot(res.resid);
1.4.3
y = X +
(1.2)
Nn (0n , 2 In )
R N (r, T )
or equivalently
(1.4)
Q N (q, Ik )
1.4. EXAMPLES
13
(, |y)
1
2
n+1
2
V 1 ( )}
exp{( )
= V / 2 (X y + 2 Q q)
where V = 2 ((X X) + 2 Q Q)1 and
Theil and Goldberger (1961) observed that the conditional posterior distribution of given 2 and y is
V )
(| 2 , y) N (,
(1.6)
2 = (y X )
k).
In a Gibbs sampling framework it is possible to simulate from the posterior by simulating iteratively (Gelfand and Smith (1990) gave a proof of the
convergence to the true posterior distribution) from the posterior conditional
distribution of given 2 and y
V )
(| 2 , y) N (,
(1.7)
( |, y) IG
n 1
, (y X) (y X)
2 2
or equivalently
(1.9)
% Simulate a dataset
n=100; k=3;
x = randn(n,k);
b = ones(k,1);
y = x*b + randn(n,1);
% Set prior
((y X) (y X)/ 2 |, y) 2n
14
r
R
T
Q
q
=
=
=
=
=
1.4. EXAMPLES
15
0.9610
11.2966
0.9200
11.2740
Theil-Goldberger estimates
1.0037
0.9569
0.9198
We apply now the inference procedure to a financial dataset. We consider
monthly data on the short-term interest rate (the three-month Treasury Bill
rate) and on the AAA corporate bond yield in the USA. As Treasury Bill
notes and AAA bonds are low-risk securities and one could expect that there
is a relationship between their interest rate. We consider data from January
1950 to December 1999.
Let yi be the monthly change in the Treasury Bill rate and zi the monthly
change in the AAA bond rate. We will fit on this set of data the heteroscedastic model presented above with
yi = 1 + 2 zi + i
that corresponds to set xi = (1, zi ) and = (1 , 2 ) in the multivariate
regression model given above. The results of the estimation procedure are
Gibb sampling estimates
Coefficient t-statistic t-probability
0.0053
0.7805
0.2177
0.2751
19.8628
Theil-Goldberger estimates
0.0057
0.2747
16
1
0.5
0
0.5
1
1.5
100
200
300
400
500
600
1.5
Residuals
1
0.5
0
0.5
1
100
200
300
400
500
600
Figure 1.1: Actual and fitted data (top) and residuals (bottom) using the
Bayesian estimates of the linear regression model.
The estimates of the 2 are 0.0283 for the Gibbs sampler and 0.0282 for the
Theil-Goldberger procedure.
The actual and fitted data and the residuals are given in Fig. 1.1. The
plot of the residuals shows that in the second half of the sample (say after
the 1975) the variance is underestimated. More precisely one should account
in the model for the time variation in the variance of the data. This call
for heteroscedastic linear regression models (see Chapter ??) or for nonlinear
models such as stochastic volatility models (see Chapter ?? and ??).
References
Gelfand, Alan E., and A.F.M Smith. 1990. Sampling-Based Approaches
to Calculating Marginal Densities, Journal of the American Statistical Association, Vol. 85, pp. 398-409.
17
Theil, Henri and Arthur S. Goldberger. 1961. On Pure and Mixed Statistical Estimation in Economics, International Economic Review, Vol. 2,
65-78.
1.5
18
//*************************************************
//
Some Pictures...
//*************************************************
// figure(1) to have distinct graphs
figure(1);
title("Time series data");
ylabel("Data");
xlabel("Time");
plot(xx,yy);
figure(2);
title("Time-varying log-volatility");
plot(xx,s,color,[1 0 0]); //[red green blue] the rgb convention
a=gca();
a.data_bounds=[1,min(s);n,max(s)];// Set tics
figure(3);
title("Dummy");
plot(xx,d,color,[1 0 0]); //[red green blue] the rgb convention
a=gca();
a.data_bounds=[1,-0.1;n,1.1];// Set tics
//*************************************************
//
All charts in one pictures...
//*************************************************
figure(4);
subplot(3,1,1);
title("Time series data");
ylabel("Data");
xlabel("Time");
plot(xx,yy);
subplot(3,1,2);
title("Time-varying log-volatility");
plot(xx,s,color,[1 0 0]); //[red green blue] the rgb convention
a=gca();
a.data_bounds=[1,min(s);n,max(s)];// Set tics
subplot(3,1,3);
title("Dummy");
plot(xx,d,color,[1 0 0]); //[red green blue] the rgb convention
a=gca();
a.data_bounds=[1,-0.1;n,1.1];// Set tics
//*************************************************
//
histogram
R
#*************************************************
# basic in I/O, graphical, statistical procedures
#*************************************************
#
Load UK/EU exchange rate data
yy=scan("C:/Dottorato/Teaching/SummerSchoolBertinoro/TutorialAntonietta/...
TutorialRobAnt/AllLab/MatlabCode/ChapterMatlab/pound.txt",sep="\t",skip=0,na.strings=".")
dim(yy)=c(1006,1);
#*************************************************
n=dim(yy);
# evaluate the number of rows #
n=n[1];
xx=(1:n);
#*************************************************
#
for endfor
if end
# (1) Evaluate sequentially the variance
# (2) Built a dummy variable, based on the value
#
of the variance estimated recursively
#*************************************************
wn=10;
# set the value of a variable#
s=array(0,n);
# define a n-dim null vector #
d=array(0,n);
for (j in ((wn+1):n)){
s[j]=var(yy[(j-wn+1):j]);
if (s[j]>0.45){
d[j]=1;
}
}
#*************************************************
#
Some Pictures...
#*************************************************
# figure(1) to have distinct graphs
19
20
dev.new();
plot(xx,yy,main="Time series data",xlab="Time",ylab="Data",type="l");
dev.new();
plot(xx,s,main="Time-varying log-volatility",xlab="Time",ylab="Data",type="l");
#[red green blue] the rgb convention
dev.new();
plot(xx,d,main="Dummy",xlab="Time",ylab="Data",type="l");
#[red green blue] the rgb convention
#*************************************************
#
All charts in one pictures...
#*************************************************
par(mfrow=c(3,1),pin=c(5,1.5));
plot(xx,yy,main="Time series data",xlab="Time",ylab="Data",type="l");
plot(xx,s,main="Time-varying log-volatility",xlab="Time",ylab="Data",type="l");
#[red green blue] the rgb convention
plot(xx,d,main="Dummy",xlab="Time",ylab="Data",type="l");
#[red green blue] the rgb convention
#*************************************************
#
histogram
#*************************************************
dev.new();
hist(yy,50);
#*************************************************
# Save the results in a ouput file
#*************************************************
save(yy, file = "C:/Dottorato/Teaching/SummerSchoolBertinoro/TutorialAntonietta/...
TutorialRobAnt/AllLab/MatlabCode/ChapterMatlab/OutPound.txt");
Chapter 2
Monte Carlo Integration
Aim
Apply basic Monte Carlo principles to solve some basic integration
problems. Discuss the choice of the number of samples in a Monte
Carlo estimation.
Contents
1. Integration
2. A Monte Carlo Estimator
3. Asymptotic Properties
4. Optimal Number of MC Samples
5. Appendix - Matlab Code
2.1
Integration
(f ) =
f (x)dx
0
21
22
2.2
(2.3)
1X
f (Xi )
n =
n i=1
23
MC Means
f(x)=x
0.8
Empirical Average
Theoretical Mean
0.6
0.4
0.2
0
10
20
30
40
50
f(x)=x
0.4
Empirical Average
Theoretical Mean
0.3
0.2
0.1
0
10
20
30
40
50
f(x)=cos( x)
1
Empirical Average
Theoretical Mean
0.5
0
0.5
10
20
30
40
50
(2.4)
(2.5)
E(
n ) =
1X
E(f (Xi )) =
n i=1
n
1 X
1
V(
n ) = 2
V(f (Xi )) = 2 (f )
n i=1
n
24
where
(f ) = V(f (X1 )) =
(x )2 f (x)I[0,1] (x)dx
For the different f we find the analytical solution of the integral (f ) (see
also horizontal dotted lines in Fig. 2.1)
1. For f (x) = x
(2.6)
E(f (X1 )) =
1 2 0
xdx = x = 1/2
2 1
2. For f (x) = x2
(2.7)
E(f (X1 )) =
2.3
E(f (X1 )) =
1
0
1
0
0
1
x2 dx = x3 = 1/3
3 1
0
1
cos(x)dx = sin(x) = 0
Asymptotic Properties
(2.9)
(2.10)
n (
n ) N (0, 2 (f ))
25
1. For f (x) = x
V(f (X1 )) = E(f (X1 )2 ) (E(f (X1 )))2
Z 1
2
Z 1
2
=
x dx
xdx
0
2. For f (x) = x2
V(f (X1 )) = 1/5 1/9 = 4/45
3. For f (x) = cos(x)
V(f (X1 )) = 1/2 0 = 1/2
When the variance V(f (X1 )) is unknown one can use the Monte Carlo
estimator
n
(2.11)
1 X
(Xi
n )2
(f ) =
n 1 i=1
2
2.4
26
0.25
Empirical Variance
Theoretical Variance
0.2
0.15
0.1
0.05
10
20
30
40
50
f(x)=x2
0.2
Empirical Variance
Theoretical Variance
0.15
0.1
0.05
0
10
20
30
40
50
f(x)=cos( x)
1.5
Empirical Variance
Theoretical Variance
1
0.5
0
10
20
30
40
50
p
2
P r |
n | (f )/n = 1
X =
n
n=
2 (f )
2
2 (f )
27
When the variance 2 (f ) is unknown one can use the Monte Carlo estimator
n2 (f ) and then apply a similar asymptotic argument. In this case the
optimal number of simulations should satisfy the following relationship
n2 (f )
(2.14)
n2
X2
n2
2
X
3. evaluate k1 = n
2 n and generate k1 samples Xn1 +1 , . . . , Xn1 +k1 (x
X
indicates the integer part of x)
Exercise: write a Matlabs code for computing the optimal number of samples that are needed to estimate (f ) for the different integrand functions f
given in Section 1 and for the accuracy level = 0.001.
2.5
28
end
%
%
%%%%%%%%% Graphics (mean) %%%%%%%%%%
figure(1);
subplot(3,1,1);
plot(gav(:,1));
line((1:n),ones(n,1)/2,color,red);
legend(Empirical Average,Theoretical Mean,...
Location,NorthEastOutside);
title(f(x)=x);
%
subplot(3,1,2);
plot(gav(:,2));
line((1:n),ones(n,1)/3,color,red);
legend(Empirical Average,Theoretical Mean,...
Location,NorthEastOutside);
title(f(x)=x^2);
%
subplot(3,1,3);
plot(gav(:,3));
line((1:n),ones(n,1)*0,color,red);
legend(Empirical Average,Theoretical Mean,...
Location,NorthEastOutside);
title(f(x)=cos(\pi x));
29
30
Chapter 3
Importance Sampling
Aim
Define and apply the importance sampling method and study its
properties.
Contents
1. Importance Sampling (IS)
2. Properties of the IS Estimators
3. Generating Student-t Variables
3.1
Importance Sampling
= E (f (X)) =
f (x)(x)dx
32
(x)
f (x)g(x)dx
g(x)
IS
n
(3.3)
1X
w(Xi )f (Xi )
=
n i=1
where
w(Xi ) =
(Xi )
,
g(Xi )
i = 1, . . . , n
3.2
! n
n
Y
1X
w(xi )f (xi )
g(xi )dxi
n i=1
i=1
(x1 )
f (x1 )g(x1 )dx1
g(x1 )
f (x1 )(x1 )dx1
33
(x1 )2
f (x1 )2 g(x1 )dx1 =
g(x1 )2
(x1 )
f (x1 )2 (x1 )dx1 <
g(x1 )
Note that if the tails of the importance density are lighter than those of the
the importance weight (x)/g(x) is not a.e. bounded and the variance of
the estimator will be infinite for many functions f .
IS
SN
n
Pn
w(Xi )f (Xi )
Pn
= i=1
i=1 w(Xi )
34
3.3
(( + 1)/2)
(x) =
(/2)
(x )2
1+
2
(+1)/2
IR (x)
sin(x)
x
5
I(x)(2.1,+)
s
x
f (x) =
1 x
3.
x5
f (x) =
I[0,+) (x)
1 + (x 3)2
1
IR (x)
(1 + ((x )/)2 )
35
2
Studentt
1
0
0
5
4
x 10
10
Normal
5
0
0
5
4
x 10
2
Cauchy
1
0
0
5
4
x 10
The inverse c.d.f. method can be applied in order to generate from the
Cauchy. If X = F 1 (U), where U U[0,1] , then X C(, ).
From the results in Fig. 3.1 one can see that the importance weights for
Student-t and Cauchy are not unstable while the importance weights associated to the normal exhibit some large jumps. For all the functions the
results in Fig. 3.2 show that the normal proposal produces jumps in the
progressive averages (green lines) that are due to the unbounded variance of
the estimator. However for the first function the normal proposal behaves
quite well when compared with the Cauchy and Student-t proposals. For the
36
second and third function the Cauchy proposal seems to converge faster than
the Student-t. In all the pictures we plotted (black lines) the approximation
obtained with an exact simulation from a Student-t with = 12.
Exercise - Use repeated Monte Carlo experiments to find the distribution
of the estimator
n (f ). Plot the 95% and 5% quantiles and the mean of the
estimator for n = 1, . . . , 50000.
The Matlab code is
%%%%%%% Importance weight for T(nustar,0,1)
function w=w1(x,nu,nustar)
w=pdf(t,x,nu)/pdf(t,x,nustar);
end
%
%%%%%%% Importance weight for N(0,nu/(nu-2))
function w=w2(x,nu)
w=pdf(t,x,nu)/pdf(normal,x,0,sqrt(nu/(nu-2)));
end
%
%%%%%%% Importance weight for C(0,1)
function w=w3(x,nu)
w=pdf(t,x,nu)/pdfcauchy(x,0,1);
end
%
clc;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Importance sampling
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
nu=12;
nustar=7;
nIS=50000;
mu1IS=zeros(nIS,4);
mu2IS=zeros(nIS,4);
mu3IS=zeros(nIS,4);
%
mu1IScum=zeros(nIS,4);
mu2IScum=zeros(nIS,4);
mu3IScum=zeros(nIS,4);
%
wIS=zeros(nIS,3);
for i=1:nIS
% Proposal 1
x1=random(t,nustar);
% Proposal 2
x2=random(normal,0,sqrt(nu/(nu-2)));
% Proposal 3
x3=tan((rand(1,1)-0.5)*pi);
%x3=random(normal,0,1)/random(normal,0,1);
% Exact
mu1IScum(:,1)=cumsum(mu1IS(:,1).*wIS(:,1))./(1:nIS);
mu1IScum(:,2)=cumsum(mu1IS(:,2).*wIS(:,2))./(1:nIS);
mu1IScum(:,3)=cumsum(mu1IS(:,3).*wIS(:,3))./(1:nIS);
mu1IScum(:,4)=cumsum(mu1IS(:,4))./(1:nIS);
%
mu2IScum(:,1)=cumsum(mu2IS(:,1).*wIS(:,1))./(1:nIS);
mu2IScum(:,2)=cumsum(mu2IS(:,2).*wIS(:,2))./(1:nIS);
mu2IScum(:,3)=cumsum(mu2IS(:,3).*wIS(:,3))./(1:nIS);
mu2IScum(:,4)=cumsum(mu2IS(:,4))./(1:nIS);
%
mu3IScum(:,1)=cumsum(mu3IS(:,1).*wIS(:,1))./(1:nIS);
mu3IScum(:,2)=cumsum(mu3IS(:,2).*wIS(:,2))./(1:nIS);
mu3IScum(:,3)=cumsum(mu3IS(:,3).*wIS(:,3))./(1:nIS);
mu3IScum(:,4)=cumsum(mu3IS(:,4))./(1:nIS);
%%
fs=14;
%%%%%%%%%%%%%%%
figure(1)
subplot(3,1,1);
plot((1:nIS),wIS(:,1));
legend(Student-t,Location,NorthEast);
set(gca,FontSize,fs);
subplot(3,1,2);
plot((1:nIS),wIS(:,2));
legend(Normal,Location,NorthEast);
set(gca,FontSize,fs);
subplot(3,1,3);
37
38
plot((1:nIS),wIS(:,3));
legend(Cauchy,Location,NorthEast);
set(gca,FontSize,fs);
figure(2)
plot((1:nIS),mu1IScum(:,1:3));
hold on;
plot((1:nIS),mu1IScum(:,4),-k);
hold off;
legend(Student-t,Normal,Cauchy,Exact,Location,NorthEast);
ylim([0.00001 0.00015]);
set(gca,FontSize,fs);
figure(3)
plot((1:nIS),mu2IScum(:,1:3));
hold on;
plot((1:nIS),mu2IScum(:,4),-k);
hold off;
legend(Student-t,Normal,Cauchy,Exact,Location,NorthEast);
ylim([1 1.4]);
set(gca,FontSize,fs);
figure(4)
plot((1:nIS),mu3IScum(:,1:3));
hold on;
plot((1:nIS),mu3IScum(:,4),-k);
hold off;
legend(Student-t,Normal,Cauchy,Exact,Location,NorthEast);
ylim([3 9]);
set(gca,FontSize,fs);
39
x 10
Studentt
Normal
Cauchy
Exact
14
12
10
8
6
4
2
0
5
4
x 10
1.4
Studentt
Normal
Cauchy
Exact
1.35
1.3
1.25
1.2
1.15
1.1
1.05
1
0
5
4
x 10
9
Studentt
Normal
Cauchy
Exact
8
7
6
5
4
3
0
5
4
x 10
40
Exercise
Importance Sampling
Consider a Student-t distribution T (, , 2 ) with density
(( + 1)/2)
(x) =
(/2)
(3.6)
(+1)/2
(x )2
1+
IR (x)
2
= E (f (X)) =
f (x)(x)dx =
(x)
f (x)g(x)dx
g(x)
sin(x)
x
41
5
I(x)(2.1,+)
42
3.
s
x
f (x) =
1 x
f (x) =
x5
I[0,+) (x)
1 + (x 3)2
Metropolis-Hastings
Write a M.-H. algorithm to generate n = 500 i.i.d. random samples from
a zero-mean and independent bivariate normal distribution, N2 (0, I2 ), with
covariance matrix, I2 and mean 0 = (0, 0) . Use alternatively independent
and random walk proposals with variance covariance matrix 2 I2 . (Try with
different values of 2 ).