Marquette University

MSCS6020

Chapter 6: The Multivariate Normal

Distribution and Copulas
Daniel B. Rowe, Ph.D.

Department of Mathematics,
Statistics, and Computer Science

Copyright 2015 by D.B. Rowe

1

Marquette University

MSCS6020

Agenda
6.1 The Multivariate Normal
6.2 Generating a Multivariate Normal
Random Vector

D.B. Rowe
2

Marquette University

MSCS6020

6.1 The Multivariate Normal-Univariate

We can obtain a random variable x that has a general
normal distribution with mean and variance 2 via the
transformation

x z .
The PDF of x can be obtained by

f ( x | , 2 ) f ( z( x)) | J ( z x) |
where z(x) is z written in terms of x and J() is the Jacobian
of the transformation.
D.B. Rowe
3

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

A bivariate (2D) PDF
of two continuous random

variables ( x1 , x2 ) depending
upon parameters satisfies

1)

0 f ( x1, x2 | ), ( x1, x2 )

2)

f ( x1 , x2 | )dx1dx2 1 .

x1x2

D.B. Rowe
4

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

x1
Let x be a 2-dimensional (or p-dimensional)
x2
random variable with PDF of x being f(x|), then
21
Marginal means.
E ( x1 | ) 1
E(x | )

E ( x2 | ) 2
Marginal variances.

21

cov( x1 , x2 | ) 12 12
var( x1 | )
var( x | )

2
var( x2 | ) 12 2
cov( x1 , x2 | )

22

Vectors are lower case, matrices are upper case.

D.B. Rowe
5

Marquette University

MSCS6020

6.1 The Multivariate Normal-Univariate

Recall: If z~N(0,1), then

mean

Note the way I

have written this.

mean

1
( z 0)(1)1 ( z 0)
1 12 z 2
1/2
f ( z)
e
2 (1)1/2 e 2
variance
2
variance

We can obtain a random variable x that has a

general normal distribution with mean and

variance 2 via the transformation x = z + .
11 11 11

11

D.B. Rowe
6

Marquette University

MSCS6020

6.1 The Multivariate Normal-Univariate

The PDF of x can be obtained by

f ( x | , ) f ( z( x)) | J ( z x) |
2

dz ( x)
J ( z x)
dx

where z=z(x) and J() is the Jacobian of the transformation.

11

The PDF of x is
11

1
f ( x | , )
e
2
2

1 x

Note the way I

have written this.

which can be written as

f ( x | , ) 2
2

1/2

2 1/2

1
( x )( 2 )1 ( x )
2

D.B. Rowe
7

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

Given two continuous random variables ( z1 , z2 ) , we write

z1
them as a 2-dimensional vector z , and this vector
21 z2
vector

has the PDF

f Z ( z | . )

vector

If z1 and z2 are independent, then

f Z ( z | ) f Z1 ( z1 | 1 ) f Z2 ( z2 | 2 ) .
D.B. Rowe
8

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

z1
Let z1 and z2 be iid N(0,1) random variables. Then, z
11
11
21 z2
has PDF f Z1 ,Z2 ( z1 , z2 ) f Z1 ( z1 ) f Z2 ( z2 )

1 12 ( z12 z22 )
.
e
2

With vector z, this can be rewritten as

21

1 12 z ' z
fZ ( z)
e
2
.
D.B. Rowe
9

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

This can also be written as

1 12 z ' z
e
2

fZ ( z)

(2 )2/2 | I 2 |1/2 e

z1
z
21 z2

mean vector
mean vector

1
( z 0) '( I 2 )1 ( z 0)
2
covariance matrix

covariance matrix

I2

and we write that z ~ N (0, I 2 ) .

21

covariance matrix

is 22
Identity matrix

mean vector

That is, the 2-dimensional random vector z has a

21

mean vector of zero and identity variance-covariance matrix.

D.B. Rowe
10

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

This means that

f Z ( z ) (2 )2/2 | I 2 |1/2 e

1
( z 0) '( I 2 )1 ( z 0)
2

E ( z1 ) 1 0
E( z)

E ( z2 ) 2 0
0

z1
z
21 z2

22 Identity matrix

21

cov( z1 , z2 ) 12 12 1 0
var( z1 )
var( z )

2
cov(
z
,
z
)
var(
z
)
0
1

1 2
2
12
2
I2
22
D.B. Rowe
11

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

If z ~ N (0, I 2 ) , then
21

f ( z ) (2 )2/2 | I 2 |1/2 e

1
( z 0) '( I 2 )1 ( z 0)
2

We can obtain a random variable21

x that has a general
normal distribution with mean vector and variance21

covariance matrix via the transformation x A z

22

21 22 21 21

where = A A, is a factorization (i.e. Cholesky or Eigen).

22 22 22

D.B. Rowe
12

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

If a random variable x has a normal distribution with
21

mean vector and variance-covariance matrix , then

21

mean vector

f ( x | , ) (2 ) p /2 | |1/2 e

mean vector

1
( x ) ' 1 ( x )
2

22

x,

p2

covariance matrix

covariance matrix

mean vector

set of pos
def matrices

covariance matrix

and we write x ~ N ( , ) . The covariance matrix , has to

21

21 22

22

be of full rank (there is an inverse in PDF).

make sure you know what this means

D.B. Rowe
13

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

Lets take a closer look at this bivariate transformation.

x A z
21 22 21 21

x1 a11 a12 z1 1

x a

2 21 a22 z2 2

x1 a11z1 a12 z2 1

x2 a21z1 a22 z2 2
We can solve for z1 and z2 in terms of x1 and x2.
11

11

11

11

D.B. Rowe
14

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

a22 a12
1
A

22
a11a22 a12a21 a21 a11

This will give us

x A z

21

22 21 21

z A1 x

21 22

21

A invertible

z1 b11 b12 x1 1
z b b x
2 21 22 2
2
21

22

b11 b12
B A

22 22
b
b
21 22
1

21

z1 b11 ( x1 1 ) b12 ( x2 2 )
z2 b21 ( x1 1 ) b22 ( x2 2 )
D.B. Rowe
15

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

z1 b11 ( x1 1 ) b12 ( x2 2 )
z2 b11 ( x1 1 ) b12 ( x2 2 )
Continuing on, this leads to
z1 ( x1 , x2 )
x1
J ( z1 , z2 x1 , x2 )
z2 ( x1 , x2 )
x1
i.e. with z B x
21

z1 ( x1 , x2 )
x2

b11 b12

z2 ( x1 , x2 ) b21 b22 22
x2

z
, the vector derivative is J B .
x 22

D.B. Rowe
16

Marquette University

MSCS6020

z B x

6.1 The Multivariate Normal-Bivariate

The distribution of vector variable x (joint of x1 and x2) is

f X ( x | ) f Z ( z( x)) | J ( z x) |
f ( z ) (2 ) p /2 | I p |1/2 e

1
( z 0) '( I p )1 ( z 0)
2

f X ( x | , ) (2 )2/2 | I p |1/2 e
AA '

21

1
B x 0 '( I p )1 B x 0

| || A || A ' || A |2 | |1/2 | A |

f X ( x | , ) (2 ) p /2 | |1/2 e

z
J
B
x 22

1
x ' 1 x
2

B A1 | |1/2 | B |
x, p
0

D.B. Rowe
17

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

This form may be more familiar

f X ( x1 , x2 | 1 , 2 , , )
2
1

2
2

1
2 1 2 1

1
Q
2

2
2

x1 1
x1 1 x2 2 x2 2
1

Q
2

2
(1 ) 1
1 2 2

1 0, 2 0, 1 1
12 / (1 2 )

12 cov( x1, x2 )

D.B. Rowe
18

Marquette University

MSCS6020

6.1 The Multivariate Normal-Bivariate

Theorem:
If x is a 2-D (or p-D) random variable from f(x|,), with

E ( x | , )

think of p=2

p1

var( x | , )

pp

then we form y A x where dimensions match

r1 rp p1 r1

and rp
A full column rank (A: rp, rp), then

E ( y | , , , A) A and var( y | , , A) A A ' .

rp p1 r1

rp pp pr

D.B. Rowe
19

Marquette University

MSCS6020

6.2 Generating a Multivariate Normal Random

Vector
Recall: Let u1~uniform(0,1) and u2~uniform(0,1).
Let z1 2ln(u1 ) cos(2 u2 ) and z2 2ln(u1 ) sin(2 u2 )
then u1 ( z1, z2 ) e

z2
1
atan .
and u2 ( z1 , z2 )
2
z1
u1 ( z1, z2 ), u2 ( z1, z2 ) | | J (u1, u2 z1, z2 ) |

1
( z12 z22 )
2

f Z1 ,Z2 ( z1, z2 | ) fU1 ,U2

1 12 z12 1 12 z22
f Z1 ,Z2 ( z1 , z2 )
e
e
2
2
This means z1 ~ N (0,1) , z2 ~ N (0,1), z1 and z2 are independent.
D.B. Rowe
20

Marquette University

MSCS6020

6.2 Generating a Multivariate Normal Random

Vector
z1
Obtain 2-D standard normal variates z z by transforming
21 2
two independent standard uniform random variates u1 and u2.
Bivariate Normal

Bivariate Normal

First half of 106 standard normal variates as z1's and second

half as z2's. Produce 5105 statistically independent z's.
D.B. Rowe
21

Marquette University

MSCS6020

6.2 Generating a Multivariate Normal Random

3

Multiplied 5105 simulated zs by A and

22
The y's are now N(,=AA).

y Az

21 22 21 21

3. 5

Bivariate Normal

3 2

21 22 2 4
added .
0.58
21
Cholesky 1.7321
0
A
22 1.1547 1.6330

x 10

Marginal
Normal

2. 5

1. 5

0. 5

-5

10

y
3. 5

x 10

y1

Marginal
Normal

2. 5

1. 5

0. 5

0
-5

5
y

10

D.B. Rowe

cross
section

y2

15

A=chol(Sigma)
22

Marquette University

MSCS6020

Homework 5:
Chapter 6: # 4, 5, 6.
* Assume Marquette Undergrads heights have
h=67 in and h=2 in while their weights have
w=150 lbs and w=4 lbs with =.75.
a) Present an algorithm for generating random height &
weights using both Cholesky and Eigen factorizations.
b) Generate 10,000 using both factorizations.
Compute summary means, variances, covariance and
correlation. Make 2D & 3D histograms.
D.B. Rowe
23

Marquette University

MSCS6020

Rowe, 2003
D.B. Rowe
24

Marquette University

MSCS6020

2
.75 2 4

4
.75 2 4

a11 2 1.4142
.75 2 4
a21
1.5
2
2
a22 4 a21
1.3229

0
1.4142
A

1.5000
1.3229

Sigma=[2,sqrt(2)*sqrt(4)*.75;sqrt(2)*sqrt(4)*.75,4];
a11=sqrt(2)
a21=.75*sqrt(2*4)/sqrt(2)
a22=sqrt(4-a21^2)
A=[a11,0;a21,a22]

D.B. Rowe
25