In near agebra, Gaussan emnaton (aso known as row reducton) s an

agorthm for sovng systems of near equatons. It s usuay understood as a

sequence of operatons performed on the assocated matrx of coemcents.
Ths method can aso be used to nd the rank of a matrx, to cacuate the
determnant of a matrx, and to cacuate the nverse of an nvertbe square
matrx. The method s named after Car Fredrch Gauss, athough t was
known to Chnese mathematcans as eary as 179 AD (see Hstory secton).
To perform row reducton on a matrx, one uses a sequence of eementary
row operatons to modfy the matrx unt the ower eft-hand corner of the
matrx s ed wth zeros, as much as possbe. There are three types of
eementary row operatons: 1) Swappng two rows, 2) Mutpyng a row by a
non-zero number, 3) Addng a mutpe of one row to another row. Usng these
operatons, a matrx can aways be transformed nto an upper tranguar
matrx, and n fact one that s n row echeon form. Once a of the eadng
coemcents (the eft-most non-zero entry n each row) are 1, and n every
coumn contanng a eadng coemcent has zeros esewhere, the matrx s
sad to be n reduced row echeon form. Ths na form s unque; n other
words, t s ndependent of the sequence of row operatons used. For
exampe, n the foowng sequence of row operatons (where mutpe
eementary operatons mght be done at each step), the thrd and fourth
matrces are the ones n row echeon form, and the na matrx s the unque
reduced row echeon form.
\eft|\begn{array}{rrr|r}
1 & 3 & 1 & 9 \\
1 & 1 & -1 & 1 \\
3 & 11 & 5 & 35
\end{array}\rght|\to
\eft|\begn{array}{rrr|r}
1 & 3 & 1 & 9 \\
0 & -2 & -2 & -8 \\
0 & 2 & 2 & 8
\end{array}\rght|\to
\eft|\begn{array}{rrr|r}
1 & 3 & 1 & 9 \\
0 & -2 & -2 & -8 \\
0 & 0 & 0 & 0
\end{array}\rght|\to
\eft|\begn{array}{rrr|r}
1 & 0 & -2 & -3 \\
0 & 1 & 1 & 4 \\
0 & 0 & 0 & 0
\end{array}\rght|
Usng row operatons to convert a matrx nto reduced row echeon form s
sometmes caed Gauss-|ordan emnaton. Some authors use the term
Gaussan emnaton to refer to the process unt t has reached ts upper
tranguar, or (non-reduced) row echeon form. For computatona reasons,
when sovng systems of near equatons, t s sometmes preferabe to stop
row operatons before the matrx s competey reduced.The process of row
reducton makes use of eementary row operatons, and can be dvded nto
two parts. The rst part (sometmes caed Forward Emnaton) reduces a
gven system to row echeon form, from whch one can te whether there are
no soutons, a unque souton, or nntey many soutons. The second part
(sometmes caed back substtuton) contnues to use row operatons unt
the souton s found; n other words, t puts the matrx nto reduced row
echeon form.
Another pont of vew, whch turns out to be very usefu to anayze the
agorthm, s that row reducton produces a matrx decomposton of the
orgna matrx. The eementary row operatons may be vewed as the
mutpcaton on the eft of the orgna matrx by eementary matrces.
Aternatvey, a sequence of eementary operatons that reduces a snge row
may be vewed as mutpcaton by a Frobenus matrx. Then the rst part of
the agorthm computes an LU decomposton, whe the second part wrtes
the orgna matrx as the product of a unquey determned nvertbe matrx
and a unquey determned reduced row echeon matrx.
Row operatons|edt|
See aso: Eementary matrx
There are three types of eementary row operatons whch may be performed
on the rows of a matrx:
Type 1: Swap the postons of two rows.
Type 2: Mutpy a row by a nonzero scaar.
Type 3: Add to one row a scaar mutpe of another.
If the matrx s assocated to a system of near equatons, then these
operatons do not change the souton set. Therefore, f one's goa s to sove
a system of near equatons, then usng these row operatons coud make the
probem easer.
Echeon form|edt|
Man artce: Row echeon form
For each row n a matrx, f the row does not consst of ony zeros, then the
eft-most non-zero entry s caed the eadng coemcent (or pvot) of that row.
So f two eadng coemcents are n the same coumn, then a row operaton of
type 3 (see above) coud be used to make one of those coemcents zero.
Then by usng the row swappng operaton, one can aways order the rows so
that for every non-zero row, the eadng coemcent s to the rght of the
eadng coemcent of the row above. If ths s the case, then matrx s sad to
be n row echeon form. So the ower eft part of the matrx contans ony
zeros, and a of the zero rows are beow the non-zero rows. The word
"echeon" s used here because one can roughy thnk of the rows beng
ranked by ther sze, wth the argest beng at the top and the smaest beng
at the bottom.
For exampe, the foowng matrx s n row echeon form, and ts eadng
coemcents are shown n red.
\eft| \begn{array}{cccc} 0&\coor{red}{\mathbf{2}}&1&-1 \\
0&0&\coor{red}{\mathbf{3}}&1 \\ 0&0&0&0 \end{array} \rght|
It s n echeon form because the zero row s at the bottom, and the eadng
coemcent of the second row (n the thrd coumn), s to the rght of the
eadng coemcent of the rst row (n the second coumn).
A matrx s sad to be n reduced row echeon form f furthermore a of the
eadng coemcents are equa to 1 (whch can be acheved by usng the
eementary row operaton of type 2), and n every coumn contanng a
eadng coemcent, a of the other entres n that coumn are zero (whch can
be acheved by usng eementary row operatons of type 3).
Exampe of the agorthm|edt|
Suppose the goa s to nd and descrbe the set of soutons to the foowng
system of near equatons:
\begn{agnat}{7}
2x &&\; + \;&& y &&\; - \;&& z &&\; = \;&& 8 & \qquad (L_1) \\
-3x &&\; - \;&& y &&\; + \;&& 2z &&\; = \;&& -11 & \qquad (L_2) \\
-2x &&\; + \;&& y &&\; +\;&& 2z &&\; = \;&& -3 & \qquad (L_3)
\end{agnat}
The tabe beow s the row reducton process apped smutaneousy to the
system of equatons, and ts assocated augmented matrx. In practce, one
does not usuay dea wth the systems n terms of equatons but nstead
makes use of the augmented matrx, whch s more sutabe for computer
manpuatons. The row reducton procedure may be summarzed as foows:
emnate x from a equatons beow L_1, and then emnate y from a
equatons beow L_2. Ths w put the system nto tranguar form. Then,
usng back-substtuton, each unknown can be soved for.
System of equatons Row operatons Augmented matrx
\begn{agnat}{7}
2x &&\; + \;&& y &&\; - \;&& z &&\; = \;&& 8 & \\
-3x &&\; - \;&& y &&\; + \;&& 2z &&\; = \;&& -11 & \\
-2x &&\; + \;&& y &&\; +\;&& 2z &&\; = \;&& -3 &
\end{agnat}
\eft| \begn{array}{ccc|c}
2 & 1 & -1 & 8 \\
-3 & -1 & 2 & -11 \\
-2 & 1 & 2 & -3
\end{array} \rght|
\begn{agnat}{7}
2x &&\; + && y &&\; - &&\; z &&\; = \;&& 8 & \\
&& && \frac{1}{2}y &&\; + &&\; \frac{1}{2}z &&\; = \;&& 1 & \\
&& && 2y &&\; + &&\; z &&\; = \;&& 5 &
\end{agnat} L_2 + \frac{3}{2}L_1 \rghtarrow L_2
L_3 + L_1 \rghtarrow L_3
\eft| \begn{array}{ccc|c}
2 & 1 & -1 & 8 \\
0 & 1/2 & 1/2 & 1 \\
0 & 2 & 1 & 5
\end{array} \rght|
\begn{agnat}{7}
2x &&\; + && y \;&& - &&\; z \;&& = \;&& 8 & \\
&& && \frac{1}{2}y \;&& + &&\; \frac{1}{2}z \;&& = \;&& 1 & \\
&& && && &&\; -z \;&&\; = \;&& 1 &
\end{agnat} L_3 + -4L_2 \rghtarrow L_3 \eft| \begn{array}{ccc|
c}
2 & 1 & -1 & 8 \\
0 & 1/2 & 1/2 & 1 \\
0 & 0 & -1 & 1
\end{array} \rght|
The matrx s now n echeon form (aso caed tranguar form)
\begn{agnat}{7}
2x &&\; + && y \;&& &&\; \;&& = \;&& 7 & \\
&& && \frac{1}{2}y \;&& &&\; \;&& = \;&& 3/2 & \\
&& && && &&\; -z \;&&\; = \;&& 1 &
\end{agnat} L_2+\frac{1}{2}L_3 \rghtarrow L_2
L_1 - L_3 \rghtarrow L_1 \eft| \begn{array}{ccc|c}
2 & 1 & 0 & 7 \\
0 & 1/2 & 0 & 3/2 \\
0 & 0 & -1 & 1
\end{array} \rght|
\begn{agnat}{7}
2x &&\; + && y \;&& &&\; \;&& = \;&& 7 & \\
&& && y \;&& &&\; \;&& = \;&& 3 & \\
&& && && &&\; z \;&&\; = \;&& -1 &
\end{agnat} 2L_2 \rghtarrow L_2
-L_3 \rghtarrow L_3 \eft| \begn{array}{ccc|c}
2 & 1 & 0 & 7 \\
0 & 1 & 0 & 3 \\
0 & 0 & 1 & -1
\end{array} \rght|
\begn{agnat}{7}
x &&\; && \;&& &&\; \;&& = \;&& 2 & \\
&& && y \;&& &&\; \;&& = \;&& 3 & \\
&& && && &&\; z \;&&\; = \;&& -1 &
\end{agnat} L_1 - L_2 \rghtarrow L_1
\frac{1}{2}L_1 \rghtarrow L_1 \eft| \begn{array}{ccc|c}
1 & 0 & 0 & 2 \\
0 & 1 & 0 & 3 \\
0 & 0 & 1 & -1
\end{array} \rght|
The second coumn descrbes whch row operatons have |ust been
performed. So for the rst step, the x s emnated from L_2 by addng
\begn{matrx}\frac{3}{2}\end{matrx} L_1 to L_2. Next x s emnated from
L_3 by addng L_1 to L_3. These row operatons are abeed n the tabe as
L_2 + \frac{3}{2}L_1 \rghtarrow L_2
L_3 + L_1 \rghtarrow L_3.
Once y s aso emnated from the thrd row, the resut s a system of near
equatons n tranguar form, and so the rst part of the agorthm s
compete. From a computatona pont of vew, t s faster to sove the
varabes n reverse order, a process known as back-substtuton. One sees
the souton s z = -1, y = 3, and x = 2. So there s a unque souton to the
orgna system of equatons.
Instead of stoppng once the matrx s n echeon form, one coud contnue
unt the matrx s n reduced row echeon form, as t s done n the tabe. The
process of row reducng unt the matrx s reduced s sometmes referred to
as Gauss-|ordan emnaton, to dstngush t from stoppng after reachng
echeon form.
Hstory|edt|
The method of Gaussan emnaton appears n the Chnese mathematca
text Chapter Eght Rectanguar Arrays of The Nne Chapters on the
Mathematca Art. Its use s ustrated n eghteen probems, wth two to ve
equatons. The rst reference to the book by ths tte s dated to 179 CE, but
parts of t were wrtten as eary as approxmatey 150 BCE.|1||2| It was
commented on by Lu Hu n the 3rd century.
The method n Europe stems from the notes of Isaac Newton.|3||4| In 1670,
he wrote that a the agebra books known to hm acked a esson for sovng
smutaneous equatons, whch Newton then supped. Cambrdge Unversty
eventuay pubshed the notes as Arthmetca Unversas n 1707 ong after
Newton eft academc fe. The notes were wdey mtated, whch made (what
s now caed) Gaussan emnaton a standard esson n agebra textbooks by
the end of the 18th century. Car Fredrch Gauss n 1810 devsed a
notaton|whch?| for symmetrc emnaton that was adopted n the 19th
century by professona hand computers to sove the norma equatons of
east-squares probems. The agorthm that s taught n hgh schoo was
named for Gauss ony n the 1950s as a resut of confuson over the hstory of
the sub|ect.|5|
Some authors use the term Gaussan emnaton to refer ony to the
procedure unt the matrx s n echeon form, and use the term Gauss-|ordan
emnaton to refer to the procedure whch ends n reduced echeon form. The
name s used because t s a varaton of Gaussan emnaton as descrbed by
Whem |ordan n 1887. However, the method aso appears n an artce by
Casen pubshed n the same year. |ordan and Casen probaby dscovered
Gauss-|ordan emnaton ndependenty.|6|
Appcatons|edt|
The hstorcay rst appcaton of the row reducton method s for sovng
systems of near equatons. Here are some other mportant appcatons of
the agorthm.
Computng determnants|edt|
To expan how Gaussan emnaton aows to compute the determnant of a
square matrx, we have to reca how the eementary row operatons change
the determnant:
Swappng two rows mutpes the determnant by -1
Mutpyng a row by a nonzero scaar mutpes the determnant by the same
scaar
Addng to one row a scaar mutpe of another does not change the
determnant.
If the Gaussan emnaton apped to a square matrx A produces a row
echeon matrx B, et d be the product of the scaars by whch the
determnant has been mutped, usng above rues. Then the determnant of
A s the quotent by d of the product of the eements of the dagona of B.
It shoud be emphaszed that, for a nn matrx, ths method needs ony O(n3)
arthmetc operatons, whe the eementary methods, usuay taught n
eementary courses, need O(2n) or O(n!) operatons. Even on the fastest
computers, the eementary methods are mpractcabe for n > 21.
Fndng the nverse of a matrx|edt|
See aso: Invertbe matrx
A varant of Gaussan emnaton caed Gauss-|ordan emnaton can be
used for ndng the nverse of a matrx, f t exsts. If A s a n by n square
matrx, then one can use row reducton to compute ts nverse matrx, f t
exsts. Frst, the n by n dentty matrx s augmented to the rght of A, formng
a n by 2n bock matrx |A | I|. Now through appcaton of eementary row
operatons, nd the reduced echeon form of ths n by 2n matrx. The matrx
A s nvertbe f and ony f the eft bock can be reduced to the dentty
matrx I; n ths case the rght bock of the na matrx s A-1. If the agorthm
s unabe to reduce the eft bock to I, then A s not nvertbe.
For exampe, consder the foowng matrx
A =
\begn{bmatrx}
2 & -1 & 0 \\
-1 & 2 & -1 \\
0 & -1 & 2
\end{bmatrx}.
To nd the nverse of ths matrx, one takes the foowng matrx augmented
by the dentty, and row reduces t as a 3 by 6 matrx:
| A | I | =
\eft| \begn{array}{rrr|rrr}
2 & -1 & 0 & 1 & 0 & 0\\
-1 & 2 & -1 & 0 & 1 & 0\\
0 & -1 & 2 & 0 & 0 & 1
\end{array} \rght|.
By performng row operatons, one can check that the reduced row echeon
form of ths augmented matrx s:
| I | B | =
\eft| \begn{array}{rrr|rrr}
1 & 0 & 0 & \frac{3}{4} & \frac{1}{2} & \frac{1}{4}\\|3pt|
0 & 1 & 0 & \frac{1}{2} & 1 & \frac{1}{2}\\|3pt|
0 & 0 & 1 & \frac{1}{4} & \frac{1}{2} & \frac{3}{4}
\end{array} \rght|.
The matrx on the eft s the dentty, whch shows A s nvertbe. The 3 by 3
matrx on the rght, B, s the nverse of A. Ths procedure for ndng the
nverse works for square matrces of any sze.
Computng ranks and bases|edt|
The Gaussan emnaton agorthm can be apped to any m \tmes n matrx
A. In ths way, for exampe, some 6 \tmes 9 matrces can be transformed to a
matrx that has a row echeon form ke
T=
\begn{bmatrx}
a & * & * & *& * & * & * & * & * \\
0 & 0 & b & * & * & * & * & * & * \\
0 & 0 & 0 & c & * & * & * & * & * \\
0 & 0 & 0 & 0 & 0 & 0 & d & * & * \\
0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & e \\
0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0
\end{bmatrx}
where the *s are arbtrary entres and a, b, c, d, e are nonzero entres. Ths
echeon matrx T contans a weath of nformaton about A: the rank of A s 5
snce there are 5 non-zero rows n T; the vector space spanned by the
coumns of A has a bass consstng of the rst, thrd, fourth, seventh and
nnth coumn of A (the coumns of a, b, c, d, e n T), and the *s te you how
the other coumns of A can be wrtten as near combnatons of the bass
coumns. Ths s a consequence of the dstrbutvty of the dot product n the
expresson of a near map as a matrx.
A of ths appes aso to the reduced row echeon form, whch s a partcuar
row echeon form.
Computatona emcency|edt|
The number of arthmetc operatons requred to perform row reducton s one
way of measurng the agorthm's computatona emcency. For exampe, to
sove a system of n equatons for n unknowns by performng row operatons
on the matrx unt t s n echeon form, and then sovng for each unknown n
reverse order, requres n(n-1) / 2 dvsons, (2n3 + 3n2 - 5n)/6
mutpcatons, and (2n3 + 3n2 - 5n)/6 subtractons,|7| for a tota of
approxmatey 2n3 / 3 operatons. Thus t has arthmetc compexty of O(n3);
see Bg O notaton. Ths arthmetc compexty s a good measure of the tme
needed for the whoe computaton when the tme for each arthmetc
operaton s approxmatvey constant. Ths s the case when the coemcents
are represented by oatng pont numbers or when they beong to a nte
ed. If the coemcents are ntegers or ratona numbers exacty represented,
the ntermedate entres can grow exponentay arge, so the bt compexty
s exponenta.|8| However, there s a varant of Gaussan emnaton, caed
Baress agorthm that avods ths exponenta growth of the ntermedate
entres, and, wth the same arthmetc compexty of O(n3), has a bt
compexty of O(n5).
Ths agorthm can be used on a computer for systems wth thousands of
equatons and unknowns. However, the cost becomes prohbtve for systems
wth mons of equatons. These arge systems are generay soved usng
teratve methods. Specc methods exst for systems whose coemcents
foow a reguar pattern (see system of near equatons).
To put an n by n matrx nto reduced echeon form by row operatons, one
needs n3 arthmetc operatons; whch s approxmatey 50% more
computaton steps.|9|
One possbe probem s numerca nstabty, caused by the possbty of
dvdng by very sma numbers. If, for exampe, the eadng coemcent of one
of the rows s very cose to zero, then to row reduce the matrx one woud
need to dvde by that number so the eadng coemcent s 1. Ths means any
error that exsted for the number whch was cose to zero woud be amped.
Gaussan emnaton s numercay stabe for dagonay domnant or
postve-dente matrces. For genera matrces, Gaussan emnaton s
usuay consdered to be stabe, when usng parta pvotng, even though
there are exampes of stabe matrces for whch t s unstabe.|10|
Generazatons|edt|
The Gaussan emnaton can be performed over any ed, not |ust the rea
numbers.
Gaussan emnaton does not generaze n any smpe way to hgher order
tensors (matrces are array representatons of order 2 tensors); even
computng the rank of a tensor of order greater than 2 s a dmcut probem.
Pseudocode|edt|
As expaned above, Gaussan emnaton wrtes a gven m n matrx A
unquey as a product of an nvertbe m m matrx S and a row-echeon
matrx T. Here, S s the product of the matrces correspondng to the row
operatons performed.
The forma agorthm to compute T from A foows. We wrte A|,|| for the entry
n row , coumn | n matrx A wth 1 beng the rst ndex. The transformaton
s performed n pace, meanng that the orgna matrx A s ost and
successvey repaced by T.
for k = 1 ... mn(m,n):
Fnd the k-th pvot:
_max := argmax ( = k ... m, abs(A|, k|))
f A|_max, k| = 0
error "Matrx s snguar!"
swap rows(k, _max)
Do for a rows beow pvot:
for = k + 1 ... m:
Do for a remanng eements n current row:
for | = k + 1 ... n:
A|, || := A|, || - A|k, || * (A|, k| / A|k, k|)
F ower tranguar matrx wth zeros:
A|, k| := 0
Ths agorthm dhers sghty from the one dscussed earer, because before
emnatng a varabe, t rst exchanges rows to move the entry wth the
argest absoute vaue to the pvot poston. Such parta pvotng mproves
the numerca stabty of the agorthm; some other varants are used.
Upon competon of ths procedure the augmented matrx w be n row-
echeon form and may be soved by back-substtuton.
Wth modern computers, Gaussan emnaton s not aways the fastest
agorthm to compute the row echeon form of matrx. There are computer
brares, ke BLAS, that expot the specctes of the computer hardware
and of the structure of the matrx to automatcay choose the best agorthm.