The Kronecker Product

The Kronecker product of a matrix A = [aij ] 2 M (m; n) and B = [bij ] 2 M (p; q) is denoted
by A B and is dened as
A

B = (aij B)ij 2 M (mp; nq) :

"

#
"
#
1 4 9
3 4
For example, if A =
and B =
, then
2 5 3
2 1
2
3
"
#
6
1B 4B 9B
62
B=
=6
46
2B 5B 3B
4

and
B

This also shows that A

In

3
4 12 16 27 36
1 8 4 18 9 7
7
7
8 15 20 9 125
2 10 5 6 3

3
2
3
12
27
4
16
36
"
#
66 15 9 8 20 127
3A 4A
7
6
A=
=6
7
42 8 18 1 4 9 5
2A 1A
4 10 6 2 5 3
B 6= B

A: Also,

3
2
B
O
7
6
..
B=4
.
5 and B
O

If Im 2 M (m) and In 2 M (n) then Im

3
2
b11 0 b12 0
60 b
0 b12 7
7
6
11
I2 = 6
7:
4b21 0 b22 0 5
0 b21 0 b22

In = Imn 2 M (mn) : Also A

O=O

A = O:

We now study some basic properties of the kronecker product in the following theorems.
Lemma 1. If A 2 M (m; n) and B 2 M (p; q) then
(1) ( A) B = A ( B) 8 2 C
(2) (A B)T = AT B T
(3) (A B) = A
B
(4) (A B) C = A (B C) for C 2 M (r; s)
(5) A (B + C) = A B + A C for A 2 M (m; n) and B; C 2 M (p; q)
(6) (A + B) C = A C + B C for A; B 2 M (m; n) and C 2 M (p; q)
(7) A B = O if and only if either A = O or B = O:

Proof: We have
( A) B = ( aij B)ij = (aij B)ij = A ( B)
This gives (1) : To prove (2), We calculate

(A

3T
a1n B
7
a2n B 7
.. 7
..
7
.
. 5
am1 B am2 B
amn B
3
a21 B T
am1 B T
7
a22 B T
am2 B T 7
T
..
.. 7
..
7 = aji B
.
.
. 5
a2n B T
amn B T

a11 B
6
6 a21 B
= (aij B)Tij = 6
6 ..
4 .

B)T

a11 B T
6
6 a12 B T
= 6
6 ..
4 .
a1n B T

a12 B
a22 B
..
.

ji

= AT

BT :

On using similar arguments we can prove (3) : Further, We have

(A

B)

a11 B
6 ..
C=4 .

C = (aij B)ij

...

am1 B

(a11 B)
6
..
= 4
.
(B

amn B
3 2
a11 (B
(a1n B) C
7 6
..
..
..
.
5=4
.
.
am1 (B
(amn B) C

(am1 B)

= A

3
a1n B
.. 7
. 5

C
C) :

C)
..

C)

This proves (4) : The relation (5) and (6) are immediate. Note that
A (B + C) = [aij (B + C)]ij = (aij B + aij C)ij = (aij B)ij + (aij C)ij = A
For the proof of (7) ; First, let either A = O or B = O: Then

2
0B
6 ..
B=4 .

0B

and
A

a11 O
6 ..
O=4 .

am1 O

implies A

B=O.

3 2
3
0B
0
0
. 7 6. .
.7
..
. .. 5 = 4 .. . . .. 5
0B

3
3 2
0
0
a1n O
.. 7 6 .. . . .. 7
. .5
. 5 = 4.

...

amn O

a1n (B
..
.

C)

amn (B

C)

B+A

7
5

C:

Conversely, let A

B = O: Then
2

3 2
3
a1n B
0
0
.. 7 6 .. . . .. 7
..
.
. .5
. 5 = 4.
amn B
0
0

a11 B
6 ..
4 .

am1 B

In order for this to be true, either B = O or aij = 0 8 i = 1; 2; : : : ; m and j = 1; 2; : : : ; n:

Lemma 2. Let A 2 M (m; n) ; B 2 M (p; q) ; C 2 M (n; k) and D 2 M (q; r) then
(A

B) (C

D) = AC

BD

Proof: Let A = [aih ] and C = [chj ] then A B = (aih B)ih and C D = (chj D)hj
By partitioned multiplication, the (i; j) block of (A B) (C D) is
n
X

n
X

aih Bchj D =

h=1

aih chj

h=1

which is the (i; j) block of AC

BD:

BD

Lemma 3. If A 2 M (n) and B 2 M (m) are normal then A

B is normal.

Proof: Using Lemma 1 and Lemma 2, We have

(A

B) (A

B) = (A
= (A

B ) (A
B) (A

B) = A A
B ) = (A

B B = AA
B) (A

BB

B) :

Lemma 4. Let A 2 M (n) and B 2 M (m) : If 2 (A) and x 2 Cn is a corresponding

eigenvector of A and if 2 (B) and y 2 Cm is a corresponding eigenvector of B then
2 (A B) and x y 2 Cnm is a corresponding eigenvector of A B: Every eigenvalue
of A B arises as such a product of eigenvalues of A and B:
Proof: We have
Ax = x and By = y
Thus
(A

B) (x

y) = Ax

By = x

y=

(x

y)

By Schurs unitary triangularization theorem, 9 unitary matrices U 2 M (n) and V 2

M (m) such that U AU = 4A and V BV = 4B ; where 4A ; 4B are upper triangular matrices
with diagonal entries 1 ; 2 ; : : : ; n and 1 ; 2 ; : : : ; m respectively then,
3

(U V ) (A B) (U V ) = (U
V ) (AU BV ) = U AU V BV = 4A 4B :
where 4A 4B is an upper triangular matrix with diagonal entries 1 1 ; 1 2 ; : : : ; n

m:

It follows from this lemma that if A and B are square matrices then, eigenvalues of A B
are same as that of B A. Hence tr (A B) = tr (B A) and det (A B) = det (B A) :
Lemma 5. Let A 2 M (n) and B 2 M (m). If
2 (A) and x 2 Cn is a corresponding
eigenvector of A and if 2 (B) and y 2 Cm is a corresponding eigenvector of B then +
is an eigenvalue of the kronecker sum (Im A) + (B In ) and y x 2 Cmn is corresponding
eigenvector.
Proof: We have
Ax = x and By = y
Thus
[(Im

A) + (B

In )] (y

x) = (Im
= y

A) (y

x) + (B

( x) + ( y)

In ) (y

x=

(y

x) = y

x) + (y

Lemma 6. If A 2 M (n) and B 2 M (m) are nonsingular then so is A

A 1 B 1:

(Ax) + (By)

x) = ( + ) (y

B and (A

B)

Proof: By Lemma 4, we get the proof of the rst part of this lemma. Also by Lemma 2,
we have
(A B) A 1 B 1 = AA 1 BB 1 = In Im = Inm :
Lemma 7. If A 2 M (n) and B 2 M (m) are unitary then A
Proof: A and B are unitary implies A
and lemma 6, we get
(A

B)

=A

= A and B

=A

Let A 2 M (m; n) : The k th Kronecker power A

= B : Therefore using Lemma 1

B = (A
is dened as A

Lemma 8. If A 2 M (n) is positive denite, then A

B is unitary.

B) :
k

=A A

(k 1)

is positive denite 8 k = 1; 2; :::n:

Proof: We will use mathematical induction to prove this lemma. For k = 2,

A

=A

; k = 2; 3; : : :

x) :

As A A is hermitian when A is hermitian. Using Lemma 4, A

Thus Result holds good for k=2. Let it holds for k=n then, we have
A

(n+1)

=A

A is positive denite.

which is positive denite.

B) = (det A)m (det B)n : Also,

Lemma 9. Let A 2 M (n) and B 2 M (m) : Then, det (A

tr (A B) = tr (A) tr (B) :
Proof: We have
det (A

B) =

mn
Y

i=1

where

are the eigenvalues of A

B. On using Lemma 4,
i

where

are the eigenvalues of A and

det (A

B) =

n Y
m
Y

k)

=
k

are the eigenvalues of B: Then

n
Y

m
Y

m
j

j=1

j=1 k=1

n
k)

= (det A)m (det B)n :

k=1

Further,
tr (A

B) =

n X
m
X

j=1 k=1

k)

n
X
j=1

m
X

= tr (A) tr (B) :

k=1

The Hadamard Product

The Hadamard product of matrix A = [aij ] 2 M (m; n) and B = [bij ] 2 M (m; n) denoted by
A B is dened as
A B = (aij bij ) 2 M (m; n) :
Thus B A = (bij aij ) = (aij bij ) = A B: Also
A (B C) = aij (bij cij ) = (aij bij ) cij = (A B) C and
A (B + C) = aij (bij + cij ) = aij bij + aij cij = A B + A C:
We now study few results on Hadamard product of matrices:
Lemma 10. If A; B 2 M (m; n) ; then
A B = (A
in which

= f1; m + 2; 2m + 3; : : : ; m2 g and
5

B) ( ; )
= f1; n + 2; 2n + 3; : : : ; n2 g :

"

#
"
#
a11 a12
b11 b12
Proof: If A =
and B =
then
a21 a22
b21 b22
2
a11 b11
"
#
6
a11 b11 a12 b12
6a11 b21
A B=
and A B = 6
4a21 b11
a21 b21 a22 b22
a21 b21
"
#
a11 b11 a12 b12
Thus(A B) [(1; 4) ; (1; 4)] =
= A B:
a21 b21 a22 b22

a11 b12
a11 b22
a21 b12
a21 b22

a12 b11
a12 b21
a22 b11
a22 b21

3
a12 b12
a12 b22 7
7
7
a22 b12 5

a22 b22

Lemma 11 . If A; B 2 M (n) are positive (semi) denite matrices then so is A B:

Proof: By Lemma 10, A B is a principal submatrix of A B: Also A B is hermitian by
Lemma 1 and have positive eigenvalues by Lemma 4. Thus using the previous Lemma, we
get the required result.

Applications
"

#
A A
Theorem 1. A matrix A is positive denite i
is positive denite.
A A
Proof: Let

1;

2; : : : ;

be the eigenvalues of A. By spectral theorem, we have

A=

1 P1

2 P2

+ ::: +

n Pn

where Pi are orthogonal projections,

Pi2
Since

= Pi

0; Pi = Pi ; Pi Pj = 0 for i 6= j and

0; therefore,

"

n
X

Pi = I:

i=1

for all i = 1; 2; : : : ; n: Since kronecker product of two positive denite matrices is positive
denite, therefore,
"
#
i

Pi

for i = 1; 2; : : : ; n: Since sum of positive denite matrices is positive, we have

n
X
i=1

"

Pi

so,
n
X
i=1

or

"

2P
n

i Pi

i Pi

i Pi

i Pi

i Pi

6i=1
4P
n

i Pi

i=1

Hence,

n
P

i Pi

i=1
n
P

i Pi

i=1

"

3
7
5

#
A A
0:
A A
"
#
"
#
1 0
A A
Conversely- The matrices
and
are positive denite. Therefore, the Hadamard
0 0
A A
product of these two matrices is positive, that is
"
# "
#
1 0
A A
0
0 0
A A
or

"

This proves A

#
A 0
0 0

0:

"

I
Theorem 2. Let A be a normal matrix. Then
A
denite.

Proof: Let

1;

2; : : : ;

A
AA

"

I A
and
A AA

are positive

be the eigenvalues of A. By spectral theorem, we have

A=

1 P1

2 P2

+ ::: +

where Pi are orthogonal projections. Now,

"
#
1
i
i

n Pn

i i

for all i = 1; 2; : : : ; n: Since kronecker product of two positive denite matrices is positive
denite, therefore,
"
#
1
i
Pi 0
i

i i

for i = 1; 2; : : : ; n: Since sum of positive denite matrices is positive, we have

"
#
n
X
1
i
Pi 0
i

i=1

so,

n
X
i=1

or

"

i i

Pi
i Pi

i Pi
i i Pi

2 P
n
n
P
Pi
6 i=1
i=1
4P
n
n
P
i Pi
i=1

Hence

i Pi
i i Pi

i=1

"

I
A

A
AA

3
7
5

0:

On the similar argument we can prove the second.

"
#
jAj A
Theorem 3. If A is normal then
is positive denite.
A jAj
Proof: Let

1;

2;

be the eigenvalues of A. By spectral theorem, we have

A=

1 P1

2 P2

n Pn

where Pi are orthogonal projections. Therefore, as we have done in the previous theorems,
we have,
"
#
j ij
i
Pi 0
j ij
i
which implies,

"
n
X
j ij
i

i=1

and hence,

"

j ij

#
jAj A
A jAj

Pi

0:

"

#
A I
Theorem 4. If A is strictly positive, then
is positive.
I A 1
Proof: Let

1;

2; : : : ;

be the eigenvalues of A. By spectral theorem, we have

A=

1 P1

2 P2

+ ::: +

n Pn

where Pi are orthogonal projections. Now, A is strictly positive implies

i = 1; 2; : : : ; n: Therefore,
"
#
1
i
Pi 0
1
1
i
"
#
n
X
1
i
which implies,
Pi 0
1
1
i
i=1
and hence,
"
#
A I
0:
I A 1
Theorem
5. If A 0 3
then
2
2
n
A
A2
1
A
A
6
7
6
6 A2
6A
A3
A2
An+1 7
6
7
6 .
..
..
.. 7 0 and 6 ..
..
6 .
.
.
.
. 5
4 .
4 .
n+1
n+2
n
n+1
2n
A
A
A A
A
Proof: Let

1;

2 ; :::;

1 P1

2 P2

where Pi are orthogonal projections. Now,

2
1
i
6
2
6 i
i
6.
..
..
6.
.
.
4.
n
i

n 6
X
6 i
6.
6.
i=1 4 .

1
6
6A
6 .
6 .
4 .

..

An An+1

2n
i

n+1
i

A
A2
..
.

n Pn

Pi

.. 7
. 5

i
2
i

..
.

+ ::: +

n
i
n+1 7
7
i
7

n+1
i

n
i

and hence,

0:

be the eigenvalues of A. By spectral theorem, we have

A=

which implies,

3
An+1
7
An+2 7
.. 7
..
7
.
. 5
A2n+1

n
i
n+1 7
7
i
7

.. 7
. 5

Pi

2n
i

3
An
7
An+1 7
.. 7
..
7
.
. 5
A2n
9

0:

> 0; for all

Also,

1
6
6A
6 .
6 .
4 .

A
A2
..
.

An An+1

3
An
7
An+1 7
.. 7
..
7
.
. 5
A2n

A
A2
..
.

A2
A3
..
.

6
6
A=6
6
4
An+1 An+2

..

3
An+1
7
An+2 7
.. 7
7
. 5

A2n+1

Remark 3. The results are not valid for discrete random variables
10

0:

References
[1]
[2] Dharmadhikari, S.W. and Joagdev, K., (1989), Upper bounds for the variance of certain
random variables, Communications in Statistics-Theory and Methods, vol. 18(9), 32353247.
[3] Seaman, J.W., Young, D.M. and Turner, D.W. (1987), On the variance of certain bounded
random variables, Math. Scientist., 12, 109-116.

11