WIDENER UNIVERSITY School of Engineering

Third Edition
Tradition & Excellence Since 1821 Mechanical Engineering

TOPIC

ME 354

04 Computations in
Mechanical Engineering

Systems of Linear
Equations

© Tulong Zhu, All rights reserved.

Systems of Linear Equations
General • The general case of a system of linear equations with n variables
Form and m equations
a11x1  a12 x2    a1n xn  b1
a21x1  a22 x2    a2n xn  b2

am1x1  am 2 x2    amn xn  bm

Matrix • The above equations can be written as the matrix form as

Form
 a11 a12  a1n   x1   b1 
a a22  a2n   x2   b2 
 21      or Ax  b
          
a  amn   xn  bm 
 m1 am 2

 Homogeneous if b = 0.
 Nonhomogeneous if b  0
Consistency • The system is said to be consistent if it has at least one solution.
• The system is inconsistent if it has no solution.
04 - 2
Linear Systems: Elementary Equation Operations
• Elementary equation operations (similar to ERO for matrices) on both
sides of the system will not change the solution(s).
General a11x1  a12 x2    a1n xn  b1 a21x1  a22 x2    a2n xn  b2
Form a21x1  a22 x2    a2n xn  b2 a11x1  a12 x2    a1n xn  b1
 
am1x1  am 2 x2    amn xn  bm am1x1  am 2 x2    amn xn  bm

Matrix  a11 a12  a1n   x1   b1   a21 a22  a2n   x1   bb12 

a a  a1n   x2   b21 
Form a22  a2n   x2   b2   11
a12
    
 21     
                     
a a  amn   xn  bm 
 m1 am 2  amn   xn  bm   m1 am 2

or use the augmented matrix Au = [A | b]

 a11 a12  a1n b1   a21 a22  a2n b2 
a a22  a2n b2  a a12  a1n b1 
 21   11 
             
a  amn bm  a  amn bm 
 m1 am 2  m1 am 2
04 - 3
What Determine(s) if Solutions Exist?
System • Ax  b
Basic Ques. • Do solutions exist ? (existence)
• If a solution exists, is it unique? (uniqueness)
• How are solutions to be found?
Observation • Obviously, the matrix A must play an important role.
 Is A the only factor to determine the existence and uniqueness
of the solution? Let’s see two examples
1 2  x1  1  1 2  x1  1
 2 4  x    2 2 4  x   1
  2      2   
 In both systems, A’s are identical, but b’s are different.
 The 1st system has solutions, but the 2nd one doesn’t.
 It can be seen that both matrices A and b play a role in
determining the existence and uniqueness of the solution.
 The augmented matrix Au = [A | b ] is always needed.
• Let’s try to solve some systems using the approach from middle
school – eliminating as many unknowns as possible in each eq.
04 - 4
What Determine(s) if Solutions Exist? Example 1
Gen. Form Ax  b A b
x1  2 x2  x3  1 1 2 1  x1  1 1 2 1 1  x1  1
1 2 3  x   1 1 2 3 1 –1R1+R2  x   1 
x1  2 x2  3x3  1   2      –1R1+R3  2  
x1  x2  3x3  1 1 1 3  x3  1 1 1 3 1  x3  0
x1  2 x2  x3  1 1 2 1  x1  1 1 2 1 1
2 x3  0 0 0 2   x   0  0 0 2 0 R23 rank(A)
  2      = rank(Au)
 x2  2 x3  0 0  1 2  x3  0 0  1 2 0
= # leading 1s
x1  2 x2  x3  1 1 2 1  x1  1 1 2 1 1 =3=n
0  1 2   x   0  0  1 2 0 –1 R2
 x2  2 x3  0   2      (1/2)R3 One unique
2 x3  0 0 0 2  x3  0 0 0 2 0
solution
x1  2 x2  x3  1 1 2 1   x1  1 1 2 1 1
0 1  2  x   0 0 1  2 0  –1R3+R1
x2  2 x3  0   The procedure
  2    2R3+R2
x3  0 0 0 1   x3  0 0 0 1 0 is called
x1  2 x2 1 1 2 0  x1  1 1 2 0 1 Gauss-Jordan
x2 0 0 1 0   x   0  0 1 0 0  –2R2+R1 Elimination
  2     
x3  0 0 0 1  x3  0 0 0 1 0 The last matrix
x1 1 1 0 0  x1  1 1 0 0 1 is the reduced
 0 1 0   x   0  0 1 0 0 
x2 0  
row echelon
  2   
04 - 5
x3  0 0 0 1  x3  0 0 0 1 0 form (rref).
What Determine(s) if Solutions Exist? Example 2
Gen. Form Ax  b A b
x1  2 x2  x3  1 1 2 1  x1  1 1 2 1 1
1 2 1  x   1 1 2 1 1 –1R1+R2
x1  2 x2  x3  1   2      –1R1+R3
x1  x2  3x3  1 1 1 3  x3  1 1 1 3 1
x1  2 x2  x3  1 1 2 1  x1  1 1 2 1 1
0 x3  0 0 0 0   x   0  0 0 0 0
  2      R23
 x2  2 x3  0 0  1 2  x3  0 0  1 2 0
x1  2 x2  x3  1 1 2 1  x1  1 1 2 1 1
0  1 2   x   0  0  1 2 0
 x2  2 x3  0   2      –R2
0 x3  0 0 0 0  x3  0 0 0 0 0
x1  2 x2  x3  1 1 2 1   x1  1 1 2 1 1
x2  2 x3  0 0 1  2  x   0 0 1  2 0 
  2      –2R2+R1
0 x3  0 0 0 0   x3  0 0 0 0 0
x1 5 x3  1 1 0 5   x1  1 1 0 5 1
x2  2 x3  0 0 1  2  x   0 0 1  2 0 
    
2  
0 x3  0 0 0 0   x3  0 0 0 0 0
 x1  1  5 x3  •Infinite numbers of solutions
x    2x 
 2  3

04 - 6
 x3   x3  •rank(A) =rank(Au) = # of leading 1’s = 2 < n = 3
What Determine(s) if Solutions Exist? Example 3
Gen. Form Ax  b A b
x1  2 x2  x3  1 1 2 1  x1  1 1 2 1 1
     1 2 1 2 –1R1+R2
x1  2 x2  x3  2 1 2 1  x2   2
  –1R1+R3
x1  x2  3x3  1 1 1 3  x3  1 1 1 3 1
x1  2 x2  x3  1 1 2 1  x1  1 1 2 1 1
0 x3  1 0 0 0  x2   1 0 0 0 1
       R23
 x2  2 x3  0 0  1 2  x3  0 0  1 2 0
x1  2 x2  x3  1 1 2 1  x1  1 1 2 1 1
     0  1 2 0 • rank(A)=2
 x2  2 x3  0 0  1 2  x2   0   –R2 • rank(Au)=3
0 x3  1 0 0 0  x3  1 0 0 0 1 • rank(A) 
x1  2 x2  x3  1 1 2 1   x1  1 1 2 1 1 rank(Au)
x2  2 x3  0 0 1  2  x2   0
0 1  2 0 
       –2R2+R1 • No solution
0 x3  1 0 0 0   x3  1 0 0 0 1
x1 5 x3  1 1 0 5   x1  1 1 0 5 1
x2  2 x3  0 0 1  2  x2   0 0 1  2 0 
       –R3+R1
0 x3  1 0 0 0   x3  1 0 0 0 1
x1 5 x3  0 1 0 5   x1  0 1 0 5 0
x2  2 x3  0 0 1  2  x2   0 0 1  2 0 
      
0 x3  1 0 0 0   x3  1 0 0 0 1
04 - 7
Rank of A Matrix: Basic Concepts
Rank • The rank of a matrix A, rank(A), is defined as the number of
linearly independent rows or columns.
 The number of independent rows equals the number of
independent columns
 rank(A)  min(m,n) if A is m  n.
 rank(A)  n for a square matrix A of size n.
 rank(A) = rank(AT)

Theorem • The elementary row operations (ERO) will not change the rank of
a matrix. The elementary row operations can lead to a simpler
matrix for which the rank can be easily obtained.

ERO:  Interchange two rows (Rij).

Review  Multiply a row by some nonzero constant (kRi).
 Add a multiple of a row to another row. (kRj + Ri)

Simple • One of the simple forms is the reduced row-echelon form (rref) of
Form the matrix, from the elementary row operations (ERO).
04 - 8
Echelon Form of a Matrix
Echelon Form • A matrix A is said to in (row) echelon form if:
 The 1st nonzero element in each row, the
leading entry, is 1. 1 0 5 7 2
 For two successive nonzero rows, the 0 0 1 1 3
 
leading 1 in the lower row is farther to the 0 0 0 1 6
right than the leading 1 in the higher row. 0 0 0 0 0
 All zero rows occur at the bottom of the 
matrix.
Reduced Row • A matrix A is said to in reduced row-
Echelon Form echelon form, rref(A), if, in addition to the 1 0 0 0 2
0 0 1 0 3
above conditions, it is also true that  
 In a column that contains the leading 0 0 0 1 6
entry of a row, all other elements are 0 0 0 0 0

zeros.
 The columns with a leading entry is called the leading
columns.
 Finding A–1 from ERO of [A | I ] is to find the reduced row-
echelon form of [A | I ].
04 - 9
Rank of a Matrix: Calculation
From rref • rank(A) = the number of leading entries in its rref(A).
Example 0 3 4 1  3 1 2 2
3 1 2 2 R12 0 3 4 1  (1/3)R1
   

1 5 2 1   
1 5 2 1  
1 1 / 3 2 / 3 2 / 3 1 1 / 3 2 / 3 2 / 3
0 3 4 1  –R1+ R3 0 3 4 1  R23
   
1 5 2 1  0 14 / 3 4 / 3 1 / 3 
1 1 / 3 2 / 3 2 / 3 1 1 / 3 2 / 3 2 / 3  –1/3R + R
0 14 / 3 4 / 3 1 / 3  (3/14) R2 0 1 2 / 7 1 / 14 2 1
    – 3R2 + R3
0 3 4 1  0 3 4 1 
1 0 4 / 7 9 / 14 1 0 0 1 / 2 
0 1 2 / 7 1 / 14  –2/7R3 + R2 0 1 0 0   Rank(A)=3
  –4/7R3 + R1  
0 0 1 1 / 4  0 0 1 1 / 4

• >> rref(A) or rrefmovie(A)

• >> rank(A)
04 - 10
Solutions: Existence
Theorem • The linear system

Recall Ex.  a11 a12  a1n   x1   b1 

a a22  a2n   x2   b2 
1, 2 & 3  21      or Ax  b
          
a  amn   xn  bm 
 m1 am 2
has solutions if and only if the coefficient matrix A and the
augmented matrix Au = [A | b ] has the same rank r , i.e.,

r  rank(A) = rank(Au)  ru

 For homogeneous, i.e., b = 0 , r  ru, and we always have

solutions (maybe one, maybe more).

 One of the solutions for a homogeneous system will be 0,

which is called the trivia solution.

 For nonhomogeneous equations, i. e., b  0, no solutions exist

if r  ru.
04 - 11
Solutions: Uniqueness
Theorem • If the linear system
 a11 a12  a1n   x1   b1 
Recall Ex. a a  a x   b 
1&2  21 22 2n   2   2 
 or Ax = b
          
a    x  b 
 m1 a m2 a mn   n   m 
has solutions (r = ru), then,
 If r = n, the system has precisely one solution.
 If r < n, the system has infinitely many solutions.
 Only r equations are independent, which can determine r
unknowns in terms of the other n – r unknowns, which can
take arbitrary values.
 These n – r arbitrary unknowns are called independent
unknowns, and the other r unknowns are dependent ones, as
they are “dependent upon the n – r independent unknowns.
 In solving the system, we can choose any n – r unknowns as
independent. However, we will later see that choosing certain
unknowns as independent will simplify the solving process.
04 - 12
Discussions on Existence and Uniqueness: m < n
Review  a11 a12  a1n   x1   b1 
a a22  a2n   x2   b2  • r = ru Solutions exist.
 21      • r = n: Solution is unique.
          
a  amn   xn  bm 
 m1 am 2

m<n • r is always smaller than n (why?).

• The system either has no solution (r  ru), or has infinite number of
solutions (r = ru < n).

Example  x1 
1 2 3   1 r = 1  ru = 2
1 2 3  x2   3
    No solution
 x3 

 x1   x1   3  3 x3 
1 2 3   1  r = ru = 2 < n,  x    1  3 x 
1 1 3  x2   2  number of solutions  2  3
   
 x3   x3   x3 

04 - 13
Discussions on Existence and Uniqueness: m = n
Review  a11 a12  a1n   x1   b1 
a a22  a2n   x2  b2  • r = ru Solutions exist.
 21      • r = n: Solution is unique.
          
a  ann   xn  bn 
 n1 an 2
m=n • r can be equal to or smaller than n.
• The system may
 have no solution (r  ru)
 have one unique solution (r = ru = n)
 have infinite number of solutions (r = ru < n).
Example 1 2  x1  1 r = 1  ru = 2
1 2  x   3
  2    No solution

1 2  x1  1 r = ru = 2 = n  x1   2
1 1   x   3 x    5 
  2   
One unique solution  2  

1 2  x1  1 r = ru = 1 < n  x1  1  2 x2  1  2

1 2  x   1 Infinite number  x    x    0    1  x2
  2     2  2     
04 - 14
of solutions
Discussions on Existence and Uniqueness: m > n
Review  a11 a12  a1n   x1   b1 
a a22  a2n   x2  b2  • r = ru Solutions exist.
 21      • r = n: Solution is unique.
          
a  ann   xn  bn 
 n1 an 2
m>n • r can be greater than, equal to or smaller than n. The system may
 have no solution (r  ru)
 have one unique solution (r  ru = n)
 have infinite number of solutions (r = ru < n).
Example 1 2 1
1 x 
5  1   3 r = 2  ru = 3
   x2    No solution
2 3 1
1 2 1 
1 x   x1   1
1   1   0 
r = ru = 2 = n
   x2    x    1 
One unique solution  2  
2 3 1
1 2  1  r = ru = 1 < n
2 4  x1   2  x1  1  2 x2  1  2
   x2   
Infinite number  x    x    0    1  x2
 2  2     
04 - 15
3 6 3 of solutions
Solutions of Linear Systems: No Solution
rref(Au) • Get the rref(Au) from the Gauss-Jordan elimination.
Solution • Get r , ru and n from the rref(Au). If r  ru, stop (no solution).
Example 1 4 1 2  x1  2 1 4 1 2 2
1 0
1 3 0 1   x2   0  3 0 1
      Au   
2 1 1 1   x3  3 2 1 1 1 3

4
 9 3
 
5  x4  5 4 9 3 5 5

x1 0 1 0 0 0 0
x2  (1 / 3) x4  0 0 1 0 1/3 0
rref( A u )   
x3  (2 / 3) x4  0 0 0 1 2/3 0

0 1 0 0 0 0 1

r = 3

 ru = 4

 r  ru, no solution!!!
04 - 16
Solutions of Linear Systems: One Solution
rref(Au) • Get r , ru and n from the rref(Au). If r = ru = n, one unique solution.
Example 1 2 3 4 1  1 2 3 4 1
 x1  4
4 5 6 7    0  5 6 7 0
  x2    
1 3 6 3     2 A u  1 3 6 3 2
 x3  
1 2 3 4   1 1 2 3 4 1 
  x4 
1 4 6 8 2 1 4 6 8 2
x1 0  1 0 0 0 0 
x2  5 / 3  0 1 0 0  5 / 3
 
x3 1 rref( A u )   0 0 1 0 1 

x4  1/ 3
 0 0 0 1 1 / 3 
00  0 0 0 0 0 

• r = ru = 4 = n  x1   0 
 x   5 / 3
• One unique solution.  2   
 x3   1 
• The solution is the first 4 x   
elements in the last column  4   1/ 3 
04 - 17
Solutions of Linear Systems: Infinite # of Solutions
rref(Au) • Get r , ru & n from rref(Au). If x1 x2 x3 x4
r = ru < n r = ru < n, infinite # of solutions.  1 0 2  6 2
 0 1 1 11 3
• Look down the columns of  
rref(A), if column j is NOT a  0 0 0 0 0
leading column, then xj is  
 0 0 0 0 0 
independent (arbitrary), otherwise
 0 0 0 0 0
xj is dependent.
• r = ru = 2 < n = 4, inf # sol.
• Each nonzero row of rref(Au)
• 2 independent: x3 & x4
represents an equation in the
reduced system, having one (and x1  2 x3  6 x4  2
only one) dependent unknown,
and all other unknowns in this
x2  x3  11 x4  3
equation (if any) independent.

• For each equation., write the x1  2  2 x3  6 x4

dependent unknown in terms of
the independent unknowns. This
is the general solution of the
x2  3  x3  11 x4
system.
04 - 18
Infinite # Solutions: One MoreExample
System 1 2 3 4 1  1 2 3 4 1
 1 4 2
4 x
5 6 7     2 
5 6 7

  x2  
1 3 6 3     2 A u  1 3 6 3 2
 x3    
1 
2 3 4    1  1 2 3 4 1 
  x4   
2 4 6 8  2 2 4 6 8 2
x1 x2 x3 x4
rref(Au)  6 0  • r = ru =3 < n = 4
 1 0 0
 0 1 0 11 0  • Infinitely many solutions.
  • Column 4 is not a leading
rref( A u )   0 0 1  4 1 / 3
  column, and x4 is an
 0 0 0 0 0  independent unknowns
 0 0 0 0 0 

Solutions  x1   6 x4   0   6   0   6 
 x    11x   0   11  0   11
 
2 4    x4      c c is any
 x3  1 / 3  4 x4  1 / 3  4  1 / 3  4  constant
x   x       0   1 
 4  4    
0 1     
04 - 19
Linear Systems with A Square A (m=n)
Matrix  a11 a12  a1n   x1   b1 
Form a a22  a2n   x2  b2  • r = ru: Solutions exist.
 21      • r = n: Solution is unique.
          
a  ann   xn  bn 
 n1 an 2

Rank of a • r  n.
Square • r = n, if det(A)  0. The matrix is said to be full ranked.
Matrix • If det(A) = 0, then r < n, and A is said to be rank deficient.
• For a full ranked matrix, rref(A) = I

Theorem • The square matrix A with a size of n is nonsingular if and only if

rank(A) = n.

 A is fully ranked = det(A) is non-zero = A is invertible

04 - 20
Homogeneous Systems with Square A (m=n)
Matrix  a11 a12  a1n   x1  0
Form a a22  a 2 n   x2   0 
 21      or Ax = 0
         
a  ann   xn  0
 n1 an 2

 There is always a trivia solution, i.e., x = 0. There maybe

more solutions depending upon the rank of A.
 If the matrix A is nonsingular, only the trivia solution
exists, since rank(A) = n for a nonsingular matrix.

 Non-trivia solutions exist if and only if the matrix A is

singular [det(A) = 0], as when A is singular, rank(A) < n.

How to • For a nonsingular A, x = 0, and no further action is needed.

Solve
• For a singular A, in addition to the trivia solution x = 0, non-trivia
solutions exist. The solution can be obtained from the rref(A).
04 - 21
An Example: Principal Stress
Problem • Find the principal stress and its corresponding direction:
  7 13  16
S   13  10 13 
 
Principal  16 13  7 
Stress • The principal stress s and its direction x satisfies Sx = s x,
or (S - s I)x  Ax = 0, i.e.,
 7  s 13  16   x1  0
 13  10  s 13   x2   0
    
  16 13  7  s   x3  0
• This system has a non-trivia solution only when det(A) = 0 :
 7 s 13  16
13  10  s 13  0
 16 13  7 s
• Expanding to give – s3 – 24s2 + 405s – 972 = 0
• The solutions are s1 = –36 , s2 = 9 and s3 = 3. These are the
04 - 22
three principal stresses.
An Example: Principal Stress - Direction
Prin. Stress  7  s 13  16   x1  0 • s1 = –36
 13  10  s 13   x2   0 • s2 = 9 and
    
  16 13  7  s   x3  0 • s3 = 3.

s1 = –36  29 13  16  x1  0 1 0  1  x1   1 

 13 26 13   x   0 rref ( A)  0 1 1   x2    1 c
  2         
 16 13 29   x3  0 0 0 0   x3   1 

s2 = 9  16 13  16  x1  0 1 0 1   x1   1
 13  19 13   x   0 rref ( A )  0 1 0 x    0  c
  2       2  
 16 13  16  x3  0 0 0 0  x3   1 

s3 = 3  10 13  16  x1  0 1 0  1   x1  1 
 13  13 13   x   0 rref ( A )  0 1  2  x2   2 c
  2         
 16 13  10  x3  0 0 0 0   x3  1 

Note • It can be easily verified that the three principal directions are
orthogonal.
04 - 23
Nonhomogeneous Systems with Square A (m=n)
Matrix  a11 a12  a1n   x1   b1 
Form a a22  a2n   x2  b2 
 21      or Ax = b
          
a  ann   xn  bn 
 n1 an 2
Cramer’s  The nonhomogeneous system has unique solution if and only if
Rule it is invertible.
Proof  r = n for an invertible matrix A.
the “if ”  n = r ru  n  ru= n=r and the system has a unique solution
 If A is singular, either there is no solution (r  ru) or there are
infinite number of solutions (r = ru < n)
How to • Use A–1
Solve • Use rref(Au) (also called Gauss-Jordan elimination).
• Gaussian elimination [similar to using rref(Au)]
• LU factorization.

04 - 24
• MATLAB©
Nonhomogeneous Systems: Use A–1
Matrix  a11 a12  a1n   x1   b1 
Form a a22  a2n   x2  b2 
 21      or Ax = b
          
a  ann   xn  bn 
 n1 an 2
Theorem • The above system has a unique solution if and only if the matrix A
is invertible, and the solution is
x = A–1b
Proof Ax  b, A 1Ax  A 1b, Ix  A 1b, x  A 1b

Example 3 5  x1  10 1  2  5 / 2
 2 4  x    8  A  
  2     1 3 / 2 
 2  5 / 2 10 0
1
xA b     
  1 3 / 2   8   2
Note • This is generally less efficient than other approaches.
• If A is singular, this method can not be used.
04 - 25
Nonhomogeneous Systems: Use rref(Au)
Conclusions • The system has a unique solution if and only if r = ru = n , and
the reduced row-echelon form of A is an identity matrix of size n.
 The solution is the last column of rref(Au).

• If r = ru < n , rows of zeros appear at the bottom of (rref(Au).

 There are infinitely many solutions.
 The solutions can be obtained from the rref(Au). The method
is the same as that used for systems with a non-square matrix
A.

• If r  ru, no solution exists.

Procedure • The procedure is the same as that for solving the linear systems
with a non-square A, as discussed previously.

Note • The method can be used for any linear system.

04 - 26
Nonhomogeneous Systems Using rref(Au): Examples
Examples 4 2 0  x1  1  1 0 0  3 / 4    3 / 4
2 1 3  x   2 rref ( A )  0 1 0 2  x 2 
    
2 u    
2 2 1  x3  3 0 0 1 1 / 2   1 / 2 

•r = ru = 3 = n, only one solution exists

4 2 0   x1  1  1 1 / 2 0 0 • r=2
2 1 3   x   2 rref ( A )  0 0 1 0 • ru = 3
  2    u  
2 1  3  x3  3 0 0 0 1 • no solution

x1 x2 x3 RHS

1 1 3   x1  2 1 1 3 2 •r = r = 1 < n
u
4 4 12  x   8  rref ( A )  0 0 0 0 •Infinite # of
  2    u  
2 2 6   x3  4 0 0 0 0 solutions
2  x2  3 x3  2  s  3t 
x x2  s 
s and t are arbitrary
   
 x3   t 
04 - 27
Gaussian Elimination
Gaussian • The Gaussian elimination is similar to the approach using rref(Au).
Elimination • The major difference is to make A an upper triangular matrix,
instead of an identity matrix when using rref(Au).
• Two stages in the approach: elimination & back substitution.
Elimination • Perform ERO on Au to make A an upper triangular matrix

a11   a1 n
a12 b1  
a11   a1 n   x1   b1 
a12
 0 a b2   0 a   x  b 
 22  a2 n  or   a 
22 2 n  2    2 
 0 0     0 0        
 0  bn   0   x  b 
 0  ann  0 0 a 
nn   n   n 

Back • The equation can be solved starting from xn, xn–1, …, x1:
Substitution b 1
xn  n xn 1  (bn 1  an 1n xn )

ann an 1n 1
1  n 
xi  
 i
b  
 ik k  i  n  1, n  2,,1
a x
aii  k i 1 

 The diagonal elements aii are called pivots.

04 - 28
Gaussian: The Elimination Procedure & Pivoting
 a11 a12  a1n b1  a11 a12  a1n b1 
ai(21)
a a22  a2 n b2  
ai1
R1  R i  0 a (1)  a2(1n) b2(1)   (1) R 2  R i
 21  a11  22 a22
            
a  i = 2,3,.., n  (1) 
 n1 an1  ann bn   0 a n1
(1)
 ann bn(1)  i = 3,4,.., n
a11 a12 a13  a1n b1  a11 a12 a13  a1n b1 
 0 a (1) b2(1)   0 a (1) b2(1) 
(1)
a23  a2(1n) (1)
 a2(1n)
 22   22 a23

 0 ( 2) ( 2)  …
0 a33  a3( 2n) b3  0 0 ( 2)
a33  a3n ( 2)
b3 ( 2) 
   
               
 0 bn( 2)  ( n 1)
a3( 2n)  ann  0 bn( n 1) 
( 2)
 0  0 0  ann
• What happens if a pivot (a diagonal element) is zero or very small?
• To avoid a zero or small pivot, switch rows to pick the absolute biggest element
down the column (from the diagonal down to the last, don’t look up!!) as the
pivot before the elimination for that column begins. This is called row pivoting.
 If no non-zero element is found, the matrix A is singular. Go to next
column. The system may have no solution or lots of solutions.

04 - 29
 Computationally, this method is more efficient than to get the rref(Au).
Gaussian Elimination Example
1 1 1   x1  3 1 1 1 3
Elimin. 
4 3 4   x2    8  A u  4 3 4
R13
     8
or   (why?)
9 3 4  x3  7  9 3 4 7 
9 3 4 7 9 3 4 7 
4 3 4 (– 4/9)R1+R2 (– 2/5)R2+R3
8 0 5 / 3 20 / 9 44 / 9 
  (– 1/9)R1+R3   No switch
1 1 1 3 0 2 / 3 5 / 9 20 / 9
9 3 4 7  9 x1  3x2  4 x3  7
0 5 / 3 20 / 9 44 / 9
  or 5 / 3x2  20 / 9 x3  44 / 9
0 0  1 / 3 4 / 15  1 / 3x3  4 / 15

Back 4 / 15 4
x3  
Subst.  1/ 3 5
 x1    1 / 5 
44 / 9  (20 / 9) x3 44 / 9  (20 / 9)(4 / 5) x    4 
x2   4  2  
5/3 5/3
 x3   4 / 5
7  3x2  4 x3 7  3(4)  4(4 / 5) 1
x1   
9 9 5
04 - 30
LU Factorization: Motivation
Motivation • It can be seen, from the Gaussian elimination method, that if the
matrix A in the linear system Ax=b is triangular, the solution
can be easily obtained.
 If A is an upper triangular matrix (as in the Gaussian
elimination) , the system can be solved from the back
substitution (solving from xn to x1).
 If A is a lower triangular matrix, the system can be solved
from the forward substitution (solving from x1 to xn).
LU • The LU factorization approach is to factor the matrix A into the
Factorization product of two triangular matrices: A=LU. L and U are the
lower and upper triangular matrices.
 The original equation Ax=b becomes LUx=b . If let Ux=y,
then Ly=b. We have two systems with triangular matrices.
 Solve for y from Ly=b from the back substitution.
 With y known, solve for x from Ux=y from the forward
substitution.
04 - 31
LU Factorization: Solution
Equations 1 0  0   y1   b1 
l 1  0   y2  b2 
Ly = b  21     
          
Ax = b  l  1   yn  bn 
 n1 ln 2
LUx = b
u11 u12  u1n   x1   y1 
y  0 u22  u2n   x2   y2 
Ux = y      
          
 0  unn   xn   yn 
 0
Forward • Solve Ly=b first for y (forward substitution):
Subs. for y i 1
y1  b1, y2  b2  l21 y1, yi  bi   lik yk i  2,, n
k 1

Back Subs. • Solve Ux=y for x (back substitution):

for x y 1
xn  n xn 1  ( yn 1  un 1n xn )
unn un 1n 1
1  n 
xi   i
y   ik k  i  n  1, n  2,,1
u x
uii  k i 1 
04 - 32
LU Factorization (Decomposition)
LU • Any nonsingular square matrix A may be decomposed into the
Factoriza- product of two triangular matrices as,
tion
A = LU

1 0  0
l 1  0
• A unit lower triangular
L   21  matrix with all diagonal
     elements equal to 1
l  1
 n1 ln 2

u11 u12  u1n 

 0 u  u2n 
U 22  • An upper triangular
      matrix.
 0  
 0 u nn 

• The factorization may not exist if no row exchanges are

performed.
• If rows are exchanged, the factorization always exits and unique.
04 - 33
LU Factorization: Procedure
Manual  a11 a12  a1n   1 0  0 u11 u12  u1n 
a a22  a2n   l21 1  0  0 u22  u2n
A   21   
                
a  ann   ln1
 n1 an 2 ln2  1  0 0  unn

How to Get • (1st row of L)  (All columns of U)  1st row of U, u1j = a1j
lij and uij?
• (2nd row of L)  (All columns of U)  2nd rows of L and U:
l21  a21 / u11, u2 j  a2 j  l21u1 j
• (ith row of L)  (All columns of U)  ith rows of L and U:
j 1 i 1
lij  (aij   lik ukj ) / u jj , uij  aij   lik ukj ,
k 1 k 1
j = 1, 2, …, i –1 j = i, i+1, …, n

 When calculating lij, ujj is divided. This will generate

computational error if |ujj| is very small.
 To reduce the error, the LU factorization with pivoting
(exchanging rows in A) should be used (MATLAB uses row
exchange!!).
04 - 34
LU Factorization: MATLAB©
>>[L,U,P]=lu(A)
• Provides the L, U and P (permutation) matrices. The
permutation matrix is basically an identity matrix with rows
and columns reordered. P reveals the information about how
rows in A are switched before the factorization is performed.
 The process from the identity matrix I to P is the same as
the process of exchanging rows in A, e.g.,
1 0 0 
P  0 0 1 
Shows that the 2nd and 3rd rows in
  A are exchanged
0 1 0

• The actual factorization in MATLAB (and it should be the way

factorization is performed to reduce errors) is
PA = LU, not A = LU

See Ex04_01.m

04 - 35
MATLAB© Solutions of Linear Systems
Operation Command Comments
inv(A)*b A–1 b For a square invertible A only.
A\b Solve Ax=b • Gives one solution only even if
more than one exists.
• Solve the system with least
square if no solution exists.
• More accurate than inv(A)*b
rref(Au) Get the rref(Au) Define Au=([A b])first.
rrefmovie(Au)
[L U P]=lu(A) LU factorization • LU factorization with pivoting.
• P is the permutation matrix,
revealing row exchange info.
rank(A) rank(A)
cond(A) Condition number • Large condition numbers
with respect to A–1 indicate a nearly singular matrix

04 - 36