M3R

Spectral Theorem

Refer to Wikipedia Content!

Conventions

Block letters like x shall be used to indicate vectors Vectors in a Cartesian space Rn shall always be represented by column vectors

We employ the transpose vT when using a row vector v We use the prime (') notation for various entities in the changed basis, such as vectors and their components

We will study change of bases during this session

Change of basis
v2 x' y' Axes need not be orthogonal! x y v1 (x, y)T or (x', y')T

Suppose M = (v1, v2) constitutes the changed basis. Then (1, 0)T in the new basis translates back to v1 in the original, and (0, 1)T translates to v2. In general, vector x' = (x', y')T in the new basis translates to x'v1 + y'v2 = M (x', y')T in the original basis i.e. x = M x'

Transformation

We know that every square matrix A represents a (linear) transformation T: x Ax Suppose we study the same transformation T under a change of basis represented by a matrix M

In the changed basis, T takes x = Mx' to the vector Ax or AMx' i.e. T: Mx' AMx'. Because the change is reversible, M is invertible. Obtain T: x' M-1AMx' Meaning, the action of A on vector v in the original basis is the same as that of M-1AM in the new basis

So if A is diagonalizable, what does this all mean?

Observation

Suppose we multiply two 2 x 2 matrices A and B. Let A = [c1 c2] as columns, and B = r1 as rows r2 It turns out that the product AB = c1 r1 + c2 r2, which is a sum of two matrices. By direct computations,

c1 r1 = a11 (b11 b12) = a11b11 a11b12 a21 a21b11 a21b12 a22b21 a22b22 c2 r2 = a12 (b21 b22) = a12b21 a12b22 a22
Add up the respective components

Expansion

Spectral Theorem (ST)

Any symmetric matrix A whose entries are real can be diagonalised by an orthogonal matrix: A = QQT That is, for a symmetric matrix A, the action of the corresponding transformation T: x Ax is that of a diagonal matrix , under a new orthonormal basis

Let A be of order n, Q = [q1 q2 qn]. Applying ST, we obtain A = [q1 q2 qn] diag(1, 2, , n) times [q1 q2 qn]T = [ 1q1 2q2 nqn ] [q1 q2 qn]T, which expands as 1q1 q1T + 2q2 q2T ++ nqn qnT

Illustration
1

2 4 5

3 5 6

Let A =

2 3

The eigenvalues of A are reordered by magnitude to obtain 1= 11.348, 2= -0.5157, and 3= 0.1709. We then compute the approximations

A1 = 1q1 q1T A2 = 1q1 q1T + 2q2 q2T A3 = ??? Use the Frobenius norm to check for goodness

A = [1 2 3; 2 4 5; 3 5 6]; [ Q, L ] = eig(A) Q = 0.7370 0.5910 0.3280 -0.7370 -0.5910 0.3280 0.3280 0.5910 0.7370

MATLAB output
Vectors and matrices have been bolded

L = -0.5157 0 0 0 0.1709 0 0 0 11.3448

>> lambda1 = L(3,3); lambda2 = L(1, 1); lambda3 = L(2, 2); >> q1 = Q(:,3); q2 = Q(:, 1); q3 = Q(:, 2); >> A1 = lambda1*q1*q1' A1 = 1.2204 2.1991 2.7422 2.1991 3.9626 4.9413 2.7422 4.9413 6.1618

>> A2 = A1 + lambda2*q2*q2' A2 = 0.9403 2.0744 2.9669 2.0744 3.9072 5.0413 2.9669 5.0413 5.9816

Frobenius norm

Let A be any m x n matrix. Then the norm given by i j | aij |2 is called the Frobenius norm

This is expressed in a simpler form as trace (AT A) In MATLAB, this becomes sqrt(sum(diag(A'*A)))

Let's compute the norm for the differences of A with the approximations A1 & A2, we get
>> D1 = sqrt(sum(diag((A1 - A)'*(A1 A)))) = 0.5433 >> D2 = sqrt(sum(diag((A2 - A)'*(A2 A)))) = 0.1709

See, the approximations are getting better!

On ellipses

The general quadratic form in x and y is given by a x2 + 2b xy + c y2 + d x + e y + g = 0 In matrix form, we have xTA x + bT x + g = 0

Here A is the symmetric matrix

a b b c

For an ellipse, discriminant D = (2b)2 4ac < 0, which means b2 ac < 0. In other words, det(A) > 0 If the ellipse is centered at (0, 0), then d = e = 0

Result

The Principal Axes Theorem

Let A be an n n symmetric matrix. Then there is an orthogonal change of variable x = Uy, that transforms the quadratic form xTA x into a quadratic form yT D y with no cross-product term

Matrix U will consist of the eigenvectors of A, which are orthogonal, and therefore can be made orthonormal

Now let's apply this result to a quadratic form representing an ellipse!

Example

Consider the ellipse discussed in class!