STR 613:

613: Advanced Ill

Ill--Conditioned Matrix
Numerical Analysis
An ill-
ill-conditioned
1 2
matrix can represent
the equations of two

Instructor lines that are almost 2 4 .0001
Dr. Ahmed Amir Khalil parallel, but not quite.

Another type of ill-
ill-
conditioned matrix is 1000 0.01 0.1
when we have matrix
values that vary by 5 0.0001 1
several degrees of 10 0.0002 10
magnitude.

For a system of linear equations, 1 2 x1 3

= No solution
there are four possibilities: 2 4 x 2 6.0001

A unique solution (consistent set of 1 2 x1 3 x1 1
equations). Two intersecting lines = Unique =
2 4.0001 x 2 6.0001 x 2 1

No solution (inconsistent set of equations).
Two parallel lines 1 2 x1 3
= x1 3

Infinite number of solutions (consistent set 2 4 .0001 x 2 6 Unique =
x2 0
of equations). Two identical lines 1 2 x1 3

The trivial solution (set of homogeneous = Infinite no. of solutions
equations, [A]{x}=0
[A]{x}=0) Two lines intersecting 2 4 x 2 6
at the origin 1 2 x1 0
= Trivial solution or eigen value
2 4 x 2 0

Gauss Jordan Example Gauss Jordan Example

5 x1 + 2 x2 + x3 = +3 5 2 1 3
4 x1 + x2 x3 = 3
4 1 1 3
2 x1 + 3 x 2 3 x3 = +5
2 3 3 5
5 2 1 3 1 0.4 0.2 0.6
4 1 1 3 4 1 1 3
2 3 3 5 2 3 3 5

1
 Gauss Jordan Example Gauss Jordan Example
1 0.4 0.2 0.6 1 0.4 0.2 0.6
4 1 1 3 0 0.6 1.8 5.4
2 3 3 5 0 3.8 2.6 6.2

1 0.4 0.2 0.6 1 0 1 3

0 0.6 1.8 5.4 0 1 3 9
0 3.8 2.6 6.2 0 0 14 28

Gauss Jordan Example Gauss Jordan Example

1 0 1 3 1 0 1 3
0 1 3 9 0 1 3 9
0 0 14 28
0 0 1 2
1 0 1 3 1 0 0 1
0 1 3 9 0 1 0 3
0 0 1 2 0 0 1 2

Gauss Jordan to get Inverse Direct Methods (continued)

5 2 1 1 0 0
Lower
Upper
4 2 1 0 1 0
2 3 3 0 0 1
? 0 0 0 ? x x x

x ? 0 0 0 ? x x
1 0 0 0 9/ 42 1/14 0
x x ? 0 0 ? x

?
0 1 0 1/ 3 13/ 42 3/14
x x x ? 0 0 0
0 0 1 1/ 3 19/ 42 1/ 4

2
 LU Decomposition
Solution is easy!

Ax=b
1 0 1 3
A=LU

LU
LUx
x=b
0 1 3 9

LetUx
U x= y
0 0 1 2
Ly=b get y

Ux=y get x

L-U Decomposition L-U Decomposition

A=LU
A=LU
a11 a12 a13 1 0 0 ? ? ?
Doolittle
a11 a12 a13 a ? 0 0 a ? ? ? a 21 a 22 a 23 = ? 1 0 0 ? ?
a a 32 a 33 ? ? 1 0 0 ?

31
a 21 a 22 a23 = ? b ? 0 0 b ? ?
a a33 ? ? c ? 0 0 c ?
a11 a12 a13 ? 1 ? ?
a 32
31 a 21 a 22 a 23 = ? ? 1 ? Crout
a
31 a 32 a 33 ? ? ? 1
Number of equations, Number of unknowns
a11 a12 a13 a ? a ? ? ?
Cholesky Method
a 21 a 22 a 23 = ? b ? b? ? Cholesky
a a 33 ? ? c ? c ?
What if A is symmetric? 31 a 32

Example using Doolittle Example using Doolittle

5 x1 + 2 x2 + x3 = +3 5 2 1
4 x1 + x 2 x3 = 3 0 3/5 9/5
2 x1 + 3 x 2 3 x3 = +5 0 19 / 5 13 / 5 R3 ( 19 / 3) R2

5 2 1
5 2 1
U = 0 3/ 5 9 / 5
4 1 1 R 2 (4 / 5) R1
2 3 3 R 3 ( 2 / 5) R1 0 0 14

3
 Example using Doolittle
L-U Decomposition
5 2 1
A=LU
U = 0 3/ 5 9 / 5 a11

a12 a13 1 0 0 ? ? ?
Doolittle
a 21 a 22 a 23 = ? 1 0 0 ? ?
0 0 14 a
31 a 32 a 33 ? ? 1 0 0 ?
a11 a12 a13 ? 1 ? ?
1 0 0
a 21 a 22 a 23 = ? ? 1 ? Crout
a
31 a 32 a 33 ? ? ? 1
L = 4/5 1 0
a11 a12 a13 a ? a ? ? ?
2 / 5 19 / 3 1
a 21 a 22 a 23 = ? b ? b? ? Cholesky
a a 32 a 33 ? ? c ? c ?
31

Choleski for a symmetric matrix Choleski for a symmetric matrix

a 11 a 12 a 13 d1 0 0 d 1 u 12 u 13
a b x 0 x z a 21 a 22 a 23 l 21 d 2 0 . 0 d 2 u 23
. a 31 a 32 a 33 l 31 l 32 d 3 0 0 d3
b d y wx 0 xw a 11 a 12 a 13 d1
2
d 1 . u 12 d 1 . u 13

2 a 21 a 22 a 23 l 21. d 1 l 21. u 12 d 2
2
l 21. u 13 d 2 . u 23
a b x x. z a 31 a 32 a 33 2
l 31. d 1 l 31. u 12 l 32. d 2 l 31. u 13 l 32. u 23 d 3

b d y .x y .z
2
x2
w

a 11 a 12 a 13 a 14 Assignment Hint: Cholesky

d1 0 0 0 d 1 c 21 c 31 c 41

for i=1
i=1:n
a 22 a 23 a 24 c 21 d 2 0 0 0 d 2 c 32 c 42  for j=i:n
. if j==1
j==1
a 33 a 34 c 31 c 32 d 3 0 0 0 d 3 c 43
 s = A(i,i); % i=1i=1, j=
j=11 is special case
c 41 c 42 c 43 d 4 0 0 0 d4 else
a 44  s = A(i,j) - C(1
C(1:i-
:i-1,i)'*C(
,i)'*C(11:i-
:i-1,j);
end
if j>i
2
d1 d 1 . c 21 d 1 . c 31 d 1 . c 41  C(i,j) = s/C(i,i);
else
2 2
c 21 d2 c 21. c 31 d 2 . c 32 c 21. c 41 d 2 . c 42  if s<=0
s<=0, error('C is not positive definite to
working precision'); end
c 31
2
c 32
2
d3
2
c 31. c 41 c 32. c 42 d 3 . c 43  C(i,i) = sqrt(s);
end
c 41
2
c 42
2
c 43
2 2
d4  end

end

4

Symmetric Matrix: A=AT
Banded Symmetric Matrix

Banded Symmetric Matrix x 0 0 x
x 0 0 x 0 0 0 0 0 0

0 x x 0 0 0 0 0 0 0 x x 0 0
x 0 0 x 0 0 0 0 0 0 0 x MS 0
x x x 0 0 0 0 0 0
x 0


0 x x 0 0 0 0 0 0 0 x 0 x x 0 x 0 0 0 0 x 0 x 0
0 x 0
x x x 0 0 0 0 0 0 0 0 0 0 x xMS
x 0 0 0
x x


x x 0 0
x 0 x x 0 x 0 0 0 0 0 0 0 x x x x 0 0 0
MS
0 0 x x x x x x x 0 x
0 0 0 x x x 0 0 0 0 0 0 0
0 0 0 0 0 0 x x 0 0
0 x 0 0
0 0 0 x x x x 0 0 0
0 0 x 0
0 0 0 0 0 0 0 0 x 0
0 0 0 0 M
x x x x 0 x 0 x
0 0 0 0 0 0 x 0 0 x 0 0
0 0 0 0 0 0 x x 0 0
(I, J) (I, J-I+1)
0 0 0 0 0 0 0 0 x 0
x
0 0 0 0 0 0 x 0 0

Banded NON
NON--Symmetric Matrix
x 0 0 x 0 0 0 0 0 0
Iterative Methods
0 0 0 x 0 0 x

x x x 0 0 0 0 0 0 0 0 0 x x x 0 0
0 x x x 0 0 0 0 0 0 0 0 x x x 0 0
Assume an initial solution x0

x 0 x x 0 x 0 0 0 0 x 0 x
Mx 0 x 0
Generate improved solution vector x1
0 0 0

0 0 0 x x x 0 0
0 0 x x x 0

Repeat until the solution converges
0 0 0 x x x x 0 0 0 0 x x x x 0 0
M
0 0 0 0 x x x x 0 x x x x x x
0 0
0 0 0 0 0 0 x x 0 0 0 0 0 x x 0 0

0 0 0 0 0 0 0 0 x 0 0 0 0 0 0 x 0
x 0 x
0 0 0 0 0 0 x 0 0 0 0 x 0 0

(I, J) (I, MS+J-I)