MS2200 Anum Part 1

Rac 1
MS2200
Numer i cal Anal y si s & Pr ogr ammi ng u e ca a y s s & og a g
Dr. I r. Rachman Set iawan
Engineering Design Cent re
( Mechanical Design Lab. )
Tel. 2500979 ext . 102
E- mail: rachmans@edc. ms. it b. ac. id
0. Lect ur es
0.1 Schedul e
Monday 14. 00 15. 00 ( 4102)
Thursday 14. 00 16. 00 ( 4102)
0.2 Ev al uat i on
Quiz - 15%
Homework/ Assignment - 20%
Mid- t est - 30%
Final t est - 35%
2 Rac 2009
0.3 Ref er ence Book
St even C. Chapra dan Raymond P. Canale, Numerical Met hods for
Engineers , Fourt h ed. , Mc Graw Hill I nt ernat ional Ed. , 2002
( available at HMM) .
Rac 2
0.4 Et cet er a
There is 5% bonus ( max. ) , proport ional t o and for min.
at t endance of 70%
Two hour Lect ure: Two- hour Lect ure:
2 part s
Break bet ween part s
Programming lab. / assignment will use MATLAB, or equivalent
freeware ( FreeMat h)
Limit : St udent s: 10 min. ; Lect urer: 15 min.
3 Rac 2009
0.5 Sy l l abus
I nt roduct ion t o Numerical Analysis, Comput ers & Basic
programming
E Errors
Root s of Polynomials & Equat ions
Mat rices & Syst em of Algebraic Equat ions
Opt imizat ion
Curve Fit t ings: I nt erpolat ion, Regression
Numerical I nt egrat ion & Different iat ion
Different ial Equat ions: ordinary part ial opt ional
4 Rac 2009
Different ial Equat ions: ordinary, part ial opt ional
Rac 3
0.6 St r uct ur e of Tex t Book
Part One: Modelling, Comput ers, and Error Analysis
Part Two: Root s of Equat ions
Part Three: Linear Algebraic Equat ions
Part Four: Opt imizat ion
Part Five: Curve Fit t ing
Part Six: Numerical Different iat ion and I nt egrat ion
Part Seven: Ordinary Different ial Equat ions
Part Eight : Part ial Different ial Equat ions
5 Rac 2009
1. I nt r o. t o Numer i cal Anal y si s
1. 1 Def i ni t i on
Numerical analysis is: y
One of t he met hods of analysis t hat consist s of
t echniques t o solve mat hemat ical equat ions wit h
arit hmet ical calculat ion
6 Rac 2009
Rac 4
Problem/
1. 2 Met hods i n Engi neer i ng Pr obl em Sol v i ng
Computer aid
Problem/
Reality
Empirical/
experiments
Simulation
/Theory
Analytical
7 Rac 2009
Computer aid
Computer aid Numerical
Solution
y
Solution
1. 3 Hi st or y
8 Rac 2009
Rac 5
9 Rac 2009
10 Rac 2009
Rac 6
1. 4 Obj ect i v es
To provide st udent s wit h a sound int roduct ion t o some
numerical met hods t o solve pract ical engineering problems
T i d i l ( ) d To int roduce programming language( s) t o st udent s, or
enhance st udent s programming skills
11 Rac 2009
2 I nt r o. t o Comput er s & Pr ogr ammi ng
2.1 Comput er s:
Elect ronic device
Takes input
Process by calculat ions
Delivers out put : numbers, graphs, sound
12 Rac 2009
Rac 7
2.2 Cat egor i es
By t he size and capabilit y:
Super comput er
Mainframe
Mini comput er
Micro comput er
Personal comput er: Desk t op, Lapt op
Palm t op comput er
Programmable calculat or
13 Rac 2009
14 Rac 2009
Rac 8
By t heir funct ion in net works:
Server
Workst at ion
Cli t Client
15 Rac 2009
By t he CPU archit ect ure
I BM compat ible
Apple
UNI X UNI X
16 Rac 2009
Rac 9
2.3 Component s
Hardware
I nput Per i pher al : Keyboard, Mouse/ t rack- ball/ t rack- point / t ouch- screen,
k di it l i d i ( ) d b b i di i kamera digit al, pemindai ( scanner) , modem, berbagai media penyimpan
dat a, mic, berbagai sensor yang dihubungkan oleh card/ PCMCI A dll.
Out put Per i pher al : Hard copy ( print er) , st orage, modem, soft copy
I / O port s: COM, PS/ 1, PS/ 2 ( ut k mouse, keyboard) , serial, paralel ( print er,
scanner, dll) , USB ( universal serial bus) , I EEE1394 ( firewire/ i- link) , infra-
red.
Cent ral Processing Unit ( CPU) : Pent iums, AMD, AMD64, Core Duo
RAM ( Random Access Memory)
Berbagai cards, a.l. Graphic, sound, modem/ LAN, special purpose cards
St orage: Hard disk
17 Rac 2009
Power supply
Mot herboard, t empat komponen a f dihubungkan melalui I / O port s,
expansion slot s ( I SA, PCI , AGP) , bus dan kabel- 2. Komponen- 2 c h
disebut sy st em uni t .
Di spl ay: monit or, LCD, TFT.
18 Rac 2009
Comp.
Rac 10
19 Rac 2009
Comp.
MB.
20 Rac 2009
Comp.
MB.
Rac 11
21 Rac 2009
Comp.
MB.
Soft ware
Operat ing syst em: Windows, DOS, Solaris, LI NUX et c.
Applicat ion: Ms Office, Aut oCAD, Ansys, Mat lab, Winamp
22 Rac 2009
Rac 12
23 Rac 2009
Brainware
Hardware manufact urer
Programmer/ soft ware builder
24 Rac 2009
Rac 13
3. Pr ogr ammi ng Basi cs
3.1 Why pr ogr ammi ng?
To t ranslat e mat hemat ical algorit hm in numerical met hods
int o a language t hat comput ers underst and
Physical problems
Mathematical model
Arithmetical model
25 Rac 2009
Numerical solution
Computer Program
Algorithm
3.2 Comput er Pr ogr ams
Comput er programs are a set of inst ruct ions t hat direct t he
comput er t o perform a cert ain t ask.
Cl ifi i Classificat ion:
High- level: Programming language e. g. Fort ran, Basic, C,
et c.
Low- level: machine language
Programming t opics:
Simple informat ion represent at ion ( const ant s, variables et c. )
Advanced informat ion reps. ( dat a st ruct ure, arrays, records)
26 Rac 2009
Mat hemat ical formulae
I nput / out put
Logical represent at ion
Modular programming
Rac 14
3.3 St r uct ur ed Pr ogr ammi ng
St ruct ured program is a set of rules t hat prescribe good
st yle habit s for t he programmer
A f S d i Apart from St ruct ured programming:
Top- down programming
Modular
Charact erist ics:
Syst emat ic, easier t o underst and
Easier t o debug, t est and modify
Requires comput ers t hat can t ranslat e it t o unst ruct ured
27 Rac 2009
Requires comput ers t hat can t ranslat e it t o unst ruct ured
version before running it
3.4 Communi cat i ons
Algorit hm: a set of st eps t o inst ruct a comput er t o perform
a cert ain t ask
Fl h i l/ hi l i f Flow- chart : a visual/ graphical represent at ion of an
algorit hm
Pseudo- code: an alt ernat ive approach t o express an
algorit hm t hat bridges t he gap bet ween flow- chart and
comput er code
28 Rac 2009
Rac 15
Comparison among t he t hree:
Inputs 1. Ask Inputs
2 Perform
Inputs
Instruction 1
Condition
?
Y
N
2. Perform
Instruction 1 to
inputs
3. If condition
satisfied then
store the result of
instruction 1
4. If condition not
satisfied then
DO
Instruction 1
IF condition THEN store
ELSE instruction 1
ENDDO
29 Rac 2010
store
redo instruction 1
with the input of
the previous
output from
instruction 1
No Symbol Meaning
1 Process
2 I nput / Out put
3 Select ion
4 Sub program
5 St art / end t erminal
6 Connect or
7 Direct ion of process
l
30 Rac 2009
8 Manual process
9 Page separat or
10 Dat a st orage
Rac 16
3.5 Pr ogr ammi ng Met hodol ogy
Planning ( algorit hm, flow- chart )
Writ ing t he code
Debugging and t est ing t he program
Making Remarks / comment ing
St oring and maint aining t he program
31 Rac 2009
3.6 Pr ogr ammi ng St r at egy
Main program is designed Top- down approach wit h st eps
easiliy underst ood
S b i d i d d l ( M d l ) i h Subprograms is designed as modules ( Modular) , wit h
det ail programming is arranged in a st ruct ured form
Bot h in main and subprograms, it is suggest ed t o use
indent at ion as a realisat ion of t he concept of st ruct ured
programming
32 Rac 2009
Rac 17
3. 7 Logi cal Repr esent at i on
Sequence
Unless direct ed ot herwise, t he comput er code is inplement ed
i t t i t t i one inst ruct ion at a t ime
Instruction 1
Instruction 2
33 Rac 2009
Instruction 3
Select ion
A means t o split t he programs flow int o branches based on t he
out come of a logical condit ion
34 Rac 2009
Rac 18
Repet it ion
A means t o perform a cert ain t ask for a number of t imes unt il a
cert ain condit ion is met .
35 Rac 2009
36 Rac 2009
Rac 19
3. 8 Pr ogr ammi ng Language
C+ +
Visual Basic Applicat ion ( VBA)
Mat lab/ SciLab Mat lab/ SciLab
37 Rac 2009
* Pr ogr ammi ng Topi cs
I nt roduct ion & familiarisat ion
Mat rices in Mat lab
Mat hemat ical Operat ion
Graphic plot t ing
38 Rac 2009
Rac 20
4. Er r or s
4.1 I nt r oduct i on
Computers
-Round-off
-Chopping
Num. Method
-Truncation
Human
-Modelling/Formulation err.
-Data uncertainty
-Blunder
39 Rac 2009
4. Er r or s
Human Error
Error on Num method
Human Error Human Error
40 Rac 2010
Human Error
Error on Computing
Rac 21
4. Er r or s
4. 2 Def i ni t i on of Er r or
Error is t he discrepancy bet ween t he t rue value and t he
approximat e value ( in t his case, generat ed from comput at ional
analysis) analysis)
Presisi
c
d
41 Rac 2009 Akurasi
a b
4. Er r or s
Precision how close t he measurement / comput at ional
result s among each ot her
Accuracy how close t he overall result s t o t he t rue value
R f i t t h h b f Referring t o t he graph before:
Figure a: precision LOW; accuracy LOW
Figure b: precision LOW; accuracy HI GH
Figure c: precision HI GH; and accuracy LOW
Figure d: precision HI GH; accuracy HI GH
42 Rac 2009
Rac 22
4. Er r or s
4.3 For mul at i on f or Er r or
x
t
: True value
x : Approximat e value
x
i
: Current approximat e value
x
i-1
: Previous approximat e value
c
t
: True percent age relat ive error
c
a
: Approximat e percent age relat ive error
% 100
t
x x
= c
43 Rac 2009
% 100
% 100
1
i
i i i
a
t
t
x
x x
x
=
=
c
c
4. Er r or : Round- of f & Choppi ng
4.4 Number s i n Comput er
Before disscussing Error due t o comput er, it is bet t er t o
disscuss How Numbers are Represent ed in Comput er.
T f b d b C Types of number represent ed by Comput er:
Base- n
n digit s
Example Base- 10:
( 8 x 10
4
) + ( 6 x 10
3
) + ( 4 x 10
2
) + ( 0 x 10
1
) + ( 9 x 10
0
) = 86,409
Example Base- 2:
101011
2
=
( 1x2
5
) + ( 0x2
4
) + ( 1x2
3
) + ( 0x2
2
) + ( 1x2
1
) + ( 1x2
0
)
44 Rac 2009
( 1x2 ) + ( 0x2 ) + ( 1x2 ) + ( 0x2 ) + ( 1x2 ) + ( 1x2 )
= ( 1x32) + ( 0x16) + ( 1x8) + ( 0x4) + ( 1x2) + ( 1x1)
= 32 + 0 + 8 + 0 + 2 + 1 = 43
10
Rac 23
Float ing point
x = ( ) m b
e
( ) = Sign
m = Mant issa, m Mant issa,
b = Base- number
e = Signed exponent
Example:
+ 0,2345 10
- 2
= 0,002345
1 0 1 1 0 1 1 1 0 0 1
Sgn of Magnitude
- 57 x 2
6
= -3648
45 Rac 2009
b = 2
e = + ( 1 x 2
2
+ 1 x 2
1
+ 0 x 2
0
) = + 6
m = - ( 1 x 2
5
+ 1 x 2
4
+ 1 x 2
3
+ 0 x 2
2
+ 0 x 2
1
+ 1 x 2
0
) = - 57
Sgn of
number
Sgn of
exponent
Magnitude
of exponent
Magnitude
of mantissa
4.5 Char act er i st i cs of Number s i n Comput er
Limit ed range of quant it ies
Finit e number of quant it ies wit hin range
I nt erval bet ween numbers
Error (computer)
46 Rac 2009
Chopping
Round-off
Rac 24
4.6 Round- of f and Choppi ng
t
3.141592654
3.1416 (Round-off)
3.1415 (Chopping)
47 Rac 2009
4. Er r or : Tr uncat i on
4.7 Tay l or Ser i es
Truncat ion error is relat ed wit h How numerical met hods
work
Review:
Funct ions / Formulae Approximat e relat ionship
Solved wit h arit hmet ical/ algebraic operat ions
Example: ( )
( )
( ) ( )
i i
i i
i
x x
x x
x f x f
x f
dx
x df
i
~ =
+
+
= 1
1
'
48 Rac 2009
One popular approach is Taylor Series
i i x x
i
+1
Rac 25
( ) ( ) ( )
( ) ( )
i i
i i i
h
x f
h
x f
h x f x f x f + + + + =
+
3 2
1
...
! 3
' ' '
! 2
' '
'
Taylor Series Expansion used in Numerical analysis:
( )
( )
( )
( )
( )
( )
i i
n
n
n
n
n i
n
x x h
h
n
f
R
R h
n
x f
=
+
=
+ +
+
+
+
1
1
1
! 1
!
! 3 ! 2
Remainder (nth term)
Step size, constant for Numerical Method
(can be adaptable in more advanced Num
49 Rac 2009
(can be adaptable in more advanced Num.

Meths).
a value of x that lies between x
i
and x
i+1
The original series has infinit e number of t erms
I n t he applicat ion of Numerical met hods, it would not
be possible Tr uncat ed
The t runcat ed series is now consist ing of Error t hat is The t runcat ed series is now consist ing of Error, t hat is
no ot her t han, t he Remainder ( R
n
) . But now it is called
Tr uncat i on Er r or
I t would not be possible nor necessary t o know exact ly
t he t runcat ion error is,
( )
1 +
=
n
n
h O R
50 Rac 2009
But it is sufficient t o know t hat t he t runcat ion error is
proport ional t o ( st ep size)
n+ 1
Rac 26
The remainder it self has infinit e t erms,
( ) ( ) ( )
( ) ( )
i i
i i i
h
x f
h
x f
h x f x f x f ...
! 3
' ' '
! 2
' '
'
3 2
1
+ + + + =
+
( ) ( ) ( )
( )
( )
( )
( )
( )
( )
( )
( )
_
m
R
n
i
n
m
i
m
m
i
m
i i i
h
n
x f
h
m
x f
h
m
x f
f f f
! 1
...
! 1 !
! 3 ! 2
1
1
1
+
+ +
+
+ +
+
+
+
t herefore needed t o be t runcat ed:
51 Rac 2009
m = finite number of term
n = infinite number of term
R
m
= Remainder after m term
( )
( )
( )
1
1
! 1
+
+
+
~
m i
m
m
h
m
x f
R
Ex ampl e
Aft T t i
( ) ( ) ( )
( ) ( )
( )
( )
( )
n i
n
i i
i i i
h
n
x f
h
x f
h
x f
h x f x f x f
! 1
...
! 3
' ' '
! 2
' '
'
3 2
1
+
+ + + + + =
+
Aft er Truncat ion:
( )
( ) ( ) ( )
( )
( ) ( )
h
R
h
x f x f
x f
R h x f x f x f
i i
i
i i i
1 1
1 1
'
'
=
+ + =
+
+
_
First-order
approximation
Truncation error
52 Rac 2009
approximation
First-derivative
(Approximated)
Rac 27
4. 8 Numer i cal di f f er ent i at i on ( Fi ni t e di vi ded di f f er ence)
Forward Difference Approximat ion of t he First Derivat ive
( )
( ) ( )
( )
( ) ( ) h O
h
f
x f
x x O
x x
x f x f
x f
i
i
i i
i i
i i
i
+
A
=
+
=
+
+
+
'
'
1
1
1
53 Rac 2009
Backward Difference Approximat ion of t he First Derivat ive
( )
( ) ( )
( )
( ) ( ) h O
h
f
x f
x x O
x x
x f x f
x f
i
i
i i
i i
i i i
i
+
V
=
+
=
+
'
'
1
1
54 Rac 2009
Derivation: Eqns: 4.19 and 4.20
Rac 28
Cent red Difference Approximat ion of t he First Derivat ive
( )
( ) ( )
( )
2 1 1
2
' h O
h
x f x f
x f
i i
i
+
=
+
More accurate
55 Rac 2009
Derivation: Eqns: 4.21 and 4.22
Finit e Difference Approximat ion of Higher Derivat ives
( ) ( ) ( )( )
( )
( ) h
x f
h x f x f x f
i
i i i
2
1 2
... 2
! 2
' '
2 '
+ +
+ + + =
( )
( ) ( ) ( )
( )
( )
( ) ( ) ( )
( )
or
h O
h
x f x f x f
x f
h O
h
x f x f x f
x f
i i i
i
i i i
i
2
2
1 1
2
1 2
2
' '
2
' '
! 2
+
+ +
+
+
=
+
+
=
Forward Diff.
56 Rac 2009
( )
( ) ( ) ( ) ( )
h
h
x f x f
h
x f x f
x f
i i i i
i
1 1
' '
+

~
Centred Diff.
More detail Fig. 23.3
Rac 29
4.9 Er r or Pr opagat i on
Error can propagat e t hrough mat hemat ical funct ions
For Funct ions of a Single Variable, t he est imat e error of
( )
t he funct ion, due t o error of variable x,
( ) ( ) ( )
( ) x x x f
x f x f x f
~ ~
'
~ ~
=
= A
( ) x f
~
A x x
~
57 Rac 2009
For Mult ivariable Funct ions
( ) x
f
x
f
x
f
x x x f
~ ~ ~ ~ ~ ~
A
c
+ + A
c
+ A
c
~ A
Example 4. 6
( )
n
n
n
x
x
x
x
x
x
x x x f ... ,..., ,
2
2
1
1
1 1
A
c
+ + A
c
+ A
c
~ A
58 Rac 2009
Rac 30
4.10 St abi l i t y and Condi t i on
Anot her applicat ion of Error propagat ion in Numerical Met hod is
St abilit y
Comput at ion is numerically unst able if t he effect of t he error of Comput at ion is numerically unst able if t he effect of t he error of
input values are grossly magnified by t he Numerical Met hod
And t he quant it y t hat represent s t he st abilit y is Condit ion:
Condit ion number 1 relat ive error in funct ion is ident ical t o
t hat of t he variable.
( )
( ) x f
x f x
Number Condition
~
~
'
~
=
59 Rac 2009
Condit ion number > 1 relat ive error of t he variable is amplified
Condit ion number < 1 relat ive error of t he variable is
at t enuat ed
Table 4. 3 & Example 4. 7
4.11 Numer i cal Er r or
The t ot al numerical error is t he Summat ion of t he
Truncat ion and Round- off errors
60 Rac 2009
Rac 31
There is no Syst emat ic and General approach
But here are a number of pract ical programming
guidelines:
U t d d i i it h t i Use ext ended- precision arit hmet ic
Avoid subt ract ing t wo nearly equal numbers, by
rearranging or reformulat e t he problem
Predict numerical error
Verificat ion / Validat ion wit h known ( t heoret ical/ empirical)
result
Tuning on some paramet ers, like st ep size, weight ing
fact ors coefficient s et c
61 Rac 2009
fact ors, coefficient s et c.
4.12 Human Er r or s
Assumpt ion/ Formulat ion Error
Dat a Uncert aint y/ Error
Blunder
62 Rac 2009
Rac 32
4.13 Char act er i st i cs of Numer i cal Met hods
Number of init ial guess
Rat e of convergence
St abilit y
Accuracy & Precision
Breadt h of applicat ion
Special Requirement s
Programming effort s required
63 Rac 2009
5. Root s of Equat i ons
F(x)
64 Rac 2009
x
x
2
= ?
x
1
= ?
F(x) = 0
Rac 33
5.2 Appl i cat i on
Fundament al Dependant I ndependent Par amet er s
Pr i nci pl e Var i abl e var i abl e
Heat balance Temperat ure Time and posit ion Thermal propert ies and
geomet ry
Mass balance Concent rat ion or
mass quant it y
Time and posit ion Chemical behaviour, mass
t ransfer coefficient s,
geomet ry
Force balance Magnit ude and
direct ion of forces
Time and posit ion St rengt h, st ruct ural
propert ies, geomet ry
Energy balance Changes in Time and posit ion Thermal propert ies mass
65 Rac 2009
Energy balance Changes in
kinet ic/ pot ent ial
energy st at es
Time and posit ion Thermal propert ies, mass,
geomet ry
Newt ons laws of
mot ions
Accelerat ion,
velocit y, or posit ion
Time and posit ion Mass, spring, damper,
geomet ry
Kirchoff laws Current and volt age Time Elect rical propert ies
5.3 Mat hemat i cal Back gr ound
Algebraic funct ion
0
1
= + + + +

f y f y f y f
n n
Polynomials as simpler form of Algebraic funct ion
Transcendent al funct ions
Logarit hmic
Exponent ial
0 ...
0 1 1
= + + + +

f y f y f y f
n n
( )
n
n n
x a x a x a a x f + + + + = ...
2
2 1 0
66 Rac 2009
Exponent ial
Trigonomet ric
et c
Rac 34
St andard met hods for locat ing t he root s of equat ions:
Det erminat ion of t he real root s of Algebraic and
Transcendent al equat ions
Single root from foreknowledge of it s approximat e locat ion Single root from foreknowledge of it s approximat e locat ion
Det erminat ion of all of real and complex root s of
polynomials
For polynomials
67 Rac 2009
5.4 Met hods
Bracket ing met hod
( Graphical)
Bisect ion
False- posit ion
Open met hod
Simple fixed- point it erat ion
Newt on- Rhapson
Secant
I ssues
68 Rac 2009
I ssues
Algorit hm
Convergence: Terminat ion crit eria, Error est imat es
Pit falls
Rac 35
5.5 Br ack et i ng Met hod
Locat ing t he root from t he change of sign by guessing
The guesses are set wit hin a range and covering t he root
i lf it self
The change in sign between
f(x
l
) and f(x
u
)
Requires algorithm to predict
the x
r
69 Rac 2009
x
l
x
u
5.5.1 ( Gr aphi cal met hod)
A good visualisat ion of Bracket ing
met hod
As a rough est imat ion t o be used As a rough est imat ion t o be used
as init ial guess for t he real
numerical met hod
Example 5. 1
70 Rac 2009
Rac 36
5.5.2 Bi sect i on met hod
Same concept as Graphical met hod, except now in a
syst emat ic way
Test :
( ) ( ) 0 < x f x f
Test :
The next predict ion is t he mid- point bet ween upper and lower
bound:
( ) ( ) 0 . <
u l
x f x f
2
u l
r
x x
x
+
=
71 Rac 2009
The rest of al gor i t hm is in Fi g. 5. 5
The basic pseudocode can be found in Fig. 5. 10, wit h t he
modificat ion as in Fig. 5. 11
x
l
x
u x
1
x
1
x
u
x
2
x
1
x
2
x
3
72 Rac 2009
x
3
x
1
x
4
Rac 37
Al gor i t hm f or Bi sect i on
73 Rac 2009
Effect of improvement on Programming
( Fig. 5. 10 vs Fig. 5. 11)
Function Bisection(input)
Iter = 0
DO
xrold=xr
xr=(xl+xu)/2
i i 1
Function Bisection(input)
Iter = 0
fl=f(xl)
DO
xrold=xr
( l )/2 iter=iter+1
IF xr ~= 0
ea=(ABS(xr-xrold)/xr)*100%
END
test=f(xl)*f(xr)
IF test<0
xu=xr
ELSEIF test>0
xl = xr
ELSE
ea=0
xr=(xl+xu)/2
fr=f(xr)
iter=iter+1
IF xr ~= 0
ea=(ABS(xr-xrold)/xr)*100%
END
test=fl*fr
IF test<0
xu=xr
ELSEIF test>0
xl = xr
74 Rac 2009
END
IF ea < es OR iter >=imax EXIT
ENDDO
Bisection = xr
END
fl = fr
ELSE
ea=0
END
IF ea < es OR iter >=imax EXIT
END
Bisection = xr
END
Rac 38
5.5.3 Fal se- posi t i on met hod
Also called as Regula Falsi or, in a more int elligent way,
Linear I nt erpolat ion Met hod
h d l l l d d The predict ion is linearly- proport ional, and expressed as:
( )( )
( ) ( )
u l
u l u
u r
x f x f
x x x f
x x

=
75 Rac 2009
The t erminat ion crit eria:
s
i
i i
a
e
x
x x
s
=
+
% 100 .
1
1
c
Pit falls of False- posit ion
i
x
+1
76 Rac 2009
Rac 39
5.6 Open Met hods
I t requires 1 ( one) init ial guess
I t converges fast er t han t he Bracket ing met hods
77 Rac 2009
x
0
x
1
However, somet imes it does not converge ( it diverges) ,
t hat is when a poor init ial guess is select ed
78 Rac 2009
x
0
x
1
Rac 40
5. 6. 1. Si mpl e f i x ed- poi nt i t er at i on
I t requires Re- formulat ion of t he problem:
( ) x f = 0
I n a numerical form
Example:
( )
( )
( )
i i
x g x
x g x
f
=
=
+1
( )
x
x
e x
x e x f
=
= = 0
79 Rac 2009
I n a numerical form:
( )
x g
i
x
i
e x

+
=
1
Graphical represent at ion: Two- curve Graphical met hod
Convergence ( Box 6. 1) :
( ) Convergent 1 ' x g < ( )
( )
( )
( )
( )
al proportion Linear
' : Error
error Oscillated 0 '
error Monotonic 0 '
Divergent 1 '
g
, 1 , i t i t
E x g E
x g
x g
x g
g
=
<
>
>
+
80 Rac 2009
p p
Rac 41
5. 6. 2 The New t on- Raphson Met hod
Arguably, t he most widely- used root locat ing met hod
I t uses a first - derivat ive relat ion, t o form a numerical algorit hm:
I t requires a pre- det ermined first derivat ive of t he funct ion
( explicit ly)
I t lacks a general convergence crit erion
The accuracy is predict ed as quadrat ic convergence:
( )
( )
i
i
i i
x f
x f
x x
'
1
=
+
81 Rac 2009
The accuracy is predict ed as quadrat ic convergence:
( )
2
1 i i
E O E =
+
Derivation: Fig. 6.5 & Eq. 6.5 6.6
5. 6. 3 The Secant Met hod
The modificat ion in t he N- R met hod, wit h t he first derivat ive
approximat ed numerically, by using Backward- finit e difference.
The numerical form is t herefore: The numerical form is, t herefore:
Or, wit h t he pre- det ermined st ep- size, ox:
( )( )
( ) ( )
i i
i i i
i i
x f x f
x x x f
x x
+
1
1
1
( )
i i
x x f o
82 Rac 2009
( )
( ) ( )
i i i
i i
i i
x f x x f
f
x x
+
=
+
o
1
Rac 42
5. 6. 4 Conv er gence
83 Rac 2009
5. 6. 5 Mul t i pl e Root s
( )
( ) ( )( )( ) 0 1 1 3
0 3 7 5 )
2 3
= =
= + =
x x x x f
x x x x f a
Some problems in finding mult iple root s of an
equat ion include:
( )
( ) ( )( )( )( )
root) triple ( 1 ; 3 : Roots
0 1 1 1 3
0 3 10 12 6 )
root) double ( 1 ; 3 : Roots
2 3 4
= =
= =
= + + =
= =
x x
x x x x x f
x x x x x f b
x x
84 Rac 2009
Bracket ing met hod: There is no change in sign
Newt on- Raphson and Secant met hods: f ( x) and f
( x) = 0
Solut ion: Ralst on & Rabinowit z met hod
Rac 43
First modificat ion of N- R:
( )
( )
i
i
i i
x f
x f
m x x
'
1
=
+
requires foreknowledge of
m, i.e. multiplicity of the root
Second modificat ion of N- R Ralst on- Rabinowit z ( Eq. 6. 13) :
( ) ( )
( ) | | ( ) ( )
i i i
i i
i i
x f x f x f
x f x f
x x
" '
'
2 1
=
+
85 Rac 2009
6. Root s of Pol y nomi al s
Example of polynomial:
( )
n
f
2
Example of polynomial in engineering:
Free- Vibrat ion problem:
( )
n
n n
x a x a x a a x f + + + + = ...
2
2 1 0
( )
0
=
= + +
rt
e x
kx x c x m ` ` `
86 Rac 2009
r
1
and r
2
??? Root - locat ing problem, of more specifically,
an Eigenvalue problem
( ) 0
2
= + +
rt
e k cr mr
Rac 44
6.2 Convent i onal Met hods
Numerical met hods are normally used t o locat e t he
complex root s of polynomials
A i l h d i i i i l As a numerical met hod using init ial guess:
Bracket ing met hods slow convergence
Open met hods possibilit y of non- convergence
( divergence)
I n t he case of complex polynomials:
Bracket ing met hod not applicable ( change of complex
sign??)
87 Rac 2009
Newt on- Raphson is more suit able ( wit h t he programming
language capabilit y of complex numbers)
6.2.1 Mul l er s met hod
Recall t he Secant met hod st raight line proj ect ion of a point t o
t he x- axis)
Mullers met hod uses parabolic proj ect ion ( higher order Muller s met hod uses parabolic proj ect ion ( higher order
approximat ion) :
I f linear approximat ion requires 2 point s ( backward finit e
difference) , parabolic Mullers met hod requires 3 point s:
[ x
0
, f(x
0
)]
[x
1
, f(x
1
)]
( ) ( ) ( ) c x x b x x a x f + + =
2
2
2 2
88 Rac 2009
[x
2
, f(x
2
)]
Rac 45
Algorit hm:
I nit ial guesses:
St ep size and first derivat ive:
2 1 0
, , x x x
( ) ( ) ( ) ( )
1 2 0 1
1 2 1 0 1 0
;
;
x f x f x f x f
x x h x x h
=
= =
o o
a, b, c :
Root est imat or:
Out put New set of parabola point s:
( )
2 1 1
0 1
0 1
; ; x f c ah b
h h
a = + =
+
= o
o o
1
1
0
0
;
h h
= = o o
ac b b
c
x x
4
2
2
2 3

+ =
1 0
x x
new
=
89 Rac 2009
And t he error:
Repeat unt il error is minimised
% 100 .
2
2 2
n
o n
a
x
x x
= c
3 2
2 1
x x
x x
new
new
=
=
Algorit hm:
I nit ial guesses:
St ep size and first derivat ive:
2 1 0
, , x x x
( ) ( ) ( ) ( )
1 2 0 1
1 2 1 0 1 0
;
;
x f x f x f x f
x x h x x h
=
= =
o o
a, b, c :
Root est imat or:
( )
2 1 1
0 1
0 1
; ; x f c ah b
h h
a = + =
+
= o
o o
1
1
0
0
;
h h
= = o o
ac b b
c
x x
4
2
2
2 3

+ =
1 0
x x
new
=
90 Rac 2009
And t he error:
Repeat unt il error is minimised
% 100 .
2
2 2
n
o n
a
x
x x
= c
3 2
2 1
x x
x x
new
new
=
=
Rac 46
The algorit hm repeat s wit h t he ever changing
unt il t he approximat e error, , reaches st opping crit eria,
.
For t he result of x for each it erat ion t he select ion of t he
2 1 0
, , x x x
a
c
s
c
For t he result of x
3
for each it erat ion, t he select ion of t he
new set of point s is governed by t he t wo general
st rat egies as follows:
I f only real root s of x
3
are locat ed, t he nearest t wo original
point are chosen
I f bot h real and complex is found,
3 2
2 1
1 0
x x
x x
x x
n
o n
o n
91 Rac 2009
6.2.2 Bai r st ow Met hod
A rat her different met hod from t he previous ones,
based on fact orisat ion of polynomials:
( )
2
f
n
Division by mononomial t erm ( x t ) yields:
Or wit h quadrat ic t erm
( )
( ) ( )( ) ( )
1 1
2
2 1 0
...
...
r x r x r x x f
x a x a x a a x f
n n n
n
n n
=
+ + + + =
( )
1 2
3 2 1 1
...

+ + + + =
n
n n
x b x b x b b x f
s rx x
2
92 Rac 2009
Or, wit h quadrat ic t erm .
s rx x
Rac 47
Bairst ow Algorit hm ( quadrat ic fact or) :
1. Choose init ial guess: r and s.
2. Calculat e b
i
s:
3. Calculat e c
i
s:
0 to 2 for
2 1
1 1
= + + =
+ =
=
+ +

n i sb rb a b
rb a b
a b
i i i i
n n n
n n
=
b
b c
n n
93 Rac 2009
4. Solve t he linear algebraic equat ion for Ar and As, and new
r and s.
1 to 2 for
2 1
1 1
= + + =
+ =
+ +

n i sc rc b c
rc b c
i i i i
n n n
Bairst ow Algorit hm ( quadrat ic fact or) , cot d. :
5. Check t he error, proceed if not yet sufficient
6. Find t he root s of
s rx x
2
94 Rac 2009
Rac 48
7. Li near Al gebr ai c Equat i ons
= + + +
n n
b
b x a x a x a
1 1 2 12 1 11
...
(
(
(

= + + +
= + + +
n
n
n n nn n n
n n
b
b
x
x
a a a
a a a
b x a x a x a
b x a x a x a
. .
2
1
2
1
2 22 21
1 12 11
2 2 1 1
2 2 2 22 1 21
...
...
...
...
(
(
(
n
b a a a
b a a a
...
...
2 2 22 21
1 1 12 11
95 Rac 2009
(
(
(
n n nn n n
n
b x a a a
. . . .
2 2
2 1
2 22 21
...
(
(
(
n nn n n
n
b a a a
b a a a
...
...
2 1
2 2 22 21
. . .
7. 2 Gauss El i mi nat i on
There are t wo st eps: Forward Eliminat ion and Back Subst it ut ion
I n t he first st ep, a Pivot equat ion is select ed, wit h t he a
ii
is t he
pivot coefficient t o find new coefficient : pivot coefficient , t o find new coefficient :
Once, complet ed t he result would be a simple, explicit relat ions
t o be solved, t hen
Formula for Back subst it ut ion:
kj
kk
ik
ij ij
a
a
a
a a =
(See pseudocode for details)
96 Rac 2009
Formula for Back subst it ut ion:
1 ..., , 2 , 1
1
1
1 1
=
=

+ =

n n i for
a
x a b
x
i
ii
n
i j
j
i
ij
i
i
i
Rac 49
7. 2. 1 Pseudocode:
DO k= 1, n- 1
DO i= k+ 1, n
fact or = a
i,k
/ a
k,k
DO j = k+ 1 n
1 13 12 11
b
b a a a
(
(
DO j = k+ 1, n
a
i,j
= a
i,j
fact or. a
k,j
ENDDO
b
i
= b
i
fact or. b
k
ENDDO
ENDDO
x
n
= b
n
/ a
n,n
DO i= n- 1,1,- 1
sum= 0
3 33
2 3 2 22
1 13 12 11
3 33 32 31
2 3 2 22 21
" " 0 0
' ' ' 0
b a
b a a
b a a a
b a a a
b a a a
(
(
(

(
(
(
Forward Elimination
97 Rac 2009
sum= 0
DO j = i+ 1,n
sum= sum+ a
i,j .
x
j
ENDDO
x
i
= ( b
i
- sum) / a
i,i
ENDDO ( )
( )
11 3 13 2 12 1 1
22 3 3 2 2 2
33 3 3
' ' '
" "
a x a x a b x
a x a b x
a b x
=
=
=

Back Substitution
7. 2. 2 Pi t f al l s of Gauss El i mi nat i on:
Division by 0 ( Pivot coefficient )
Round- off error
I ll- condit ion ( Fig. 9. 2) , due t o:
Mat rix Det erminant 0
Mat rix det erminant = 0 singular, infinit e solut ion
98 Rac 2009
Rac 50
7.2.3 I mpr ov ement s
More significant figure, t o resolve round- off error problem
Pivot ing st rat egy, t o avoid division by zero ( Example 9. 9)
The equat ion wit h t he largest coefficient shall be t he
pivot ing equat ion
Scaling, t o resolve round- off error problem ( Example 9. 10)
Modifying t he order of magnit ude of one or more
equat ions so t hat all equat ions have roughly similar order
of magnit ude
I ll- condit ioned problems are much more difficult and
d d l
99 Rac 2009
requires advanced t reat ment . However, most real
engineering problems normally well- condit ioned, so if you
find ill- condit ion problems check and recheck t he physical
formulat ion
7.3 Gauss- Jor dan
Modificat ion of Gauss- eliminat ion
The first st ep is modified, so t hat all unknown is eliminat ed
h h j h b Thi l i rat her t han j ust t he subsequent one. This result s in an
I dent it y mat rix
Hence, t here is no need for Back- subst it ut ion
But , t he eliminat ion st ep requires even more work,
result ing in more comput at ion t han Gauss met hod, in
t ot al.
100 Rac 2009
Rac 51
Algorit hm for Gauss- Jordan:
Normalise t he first row by t he
coefficient a
11
(
(
(
(
2 3 2 22 21
1 13 12 11
b a a a
b a a a
Wit h t he use of t he normalised
version of t he fisrt row,
eliminat e t he first column of
each of t he remaining rows ( a
i1
)
Normalise a
22
Eliminat e a
i2

and a
32
And so on unt il t he last column

(
(
1
3 33 32 31
" " 0 0 1
process
n eliminatio Forward
b
b a a a
101 Rac 2009
And so on unt il t he last column
and row ( a
mn
)
(
(
(
3
2
" " 1 0 0
" " 0 1 0
b
b
Comparison wit h Gauss Eliminat ion:
Concept ually simpler ( no apparent back subst it ut ion
process)
All pit falls in Gauss eliminat ion prevails All pit falls in Gauss eliminat ion prevails
But , relat ively more comput at ion ( approx. 50% more)
St ill used in some numerical met hods.
102 Rac 2009
Rac 52
* Fav or i t e Pr obl ems
9. 7
9. 9
P9. 14
103 Rac 2009
7. 4 LU Decomposi t i on
(
(
(
(
(
(
0 1
0 0 1
0
21 23 22
13 12 11
l u u
u u u
The Forward eliminat ion st age, bot h in Gauss and Gauss- Jordan
is a t ime- consuming process
Wit h LU Decomposit ion t his st age is t ried t o be simplified by
(
(
(
(
1
0
0 0
0
32 31
21
33
23 22
l l
l
u
u u
Upper matrix Lower matrix
104 Rac 2009
Wit h LU Decomposit ion, t his st age is t ried t o be simplified, by
not comput ing t he right - hand side of equat ions.
This, part icularly beneficial if we have t he same syst em wit h t he
same [ A] but different set of { b} . Example: I n FE model, we
have t he same shape of st ruct ure but different force syst em.
Rac 53
Descr i pt i on of Pr ocess:
| | a a a
a a a
A
(
(
(
=
23 22 21
13 12 11
| | | | L U
| |
| |
| |
f f
f
l l
l L
a
a a
a a a
u
u u
u u u
U
a a a
a a a A
(
(
(
=
(
(
(
=
(
(
(
=
(
(
(
=
(
(
1
0 1
0 0 1
1
0 1
0 0 1
" 0 0
' ' 0
0 0
0
21 21
33
23 22
13 12 11
33
23 22
13 12 11
33 32 31
23 22 21
105 Rac 2009
| || | | | A U L
f f l l
=
(
: check
1 1
32 31 32 31
Pr ogr ammi ng St r at egy :
The Decompose process is t o form a mat rix [ A] wit h t he
following example form, So t hat , t he comput er only st ores
one mat rix but will be used according t o t he procedure [ U] ,
[ L] : [ L] :
SUB Decompose ( a, n)
DO k = 1, n- 1
DO i = k+ 1, n
fact or= a
i, k
/ a
k, k
a
i, k
= fact or
DO j k+ 1
| |
(
(
(
(
=
nn n n
n
u l l
u u l
u u u
A
. . .
2 1
23 22 21
1 12 11
'
106 Rac 2009
DO j = k+ 1, n
a
i, j
= a
i, j
fact or* a
k, j
ENDDO
ENDDO
ENDDO
END Decompose
Rac 54
Basi c Al gor i t hm:
During eliminat ion phase, fact ors t hat are calculat ed are
st ored as l
ij
P i l i i i i l d i d f h Part ial pivot ing st rat egy is implement ed, inst ead of t he
whole row
While t he equat ions are not scaled, scaling is used t o
det ermine if pivot ing is t o be implement ed
The diagonal t erms are monit ored for near- zero
occurances in order t o raise singularit y warning.
107 Rac 2009
The Subst it ut ion Process is by obt aining { D} and finally { X} ,
using t he following relat ionships:
1
l d
n
1 2 1
, , 3 , 2
1
1
1
=
=
= =
=
+ =
=
n n i for
x a d
x
a d x
n i for b a d d
l d
n
i j
j ij i
nn n n
i
j
j ij i i
n
nn n
108 Rac 2009
1 , , 2 , 1 n n i for
a
x
ii
i
Rac 55
Ex ampl e 10.1 10.2
| |
(
(
(
(
(
(

= 293333 . 0 00333 . 7 0
2 . 0 1 . 0 3
3 . 0 7 1 . 0
2 . 0 1 . 0 3
A
| |
= =
= =
=
(
(
(
(
(
(
(

10000000 . 0
3
3 . 0
03333333 . 0
3
1 . 0
0120 . 10 0 0
293333 . 0 00333 . 7 0
2 . 0 1 . 0 3
02 . 10 19 . 0 0 10 2 . 0 3 . 0
31
21
f
f
U
109 Rac 2009
| |
(
(
(
=
=
=
1 0271300 . 0 10000000 . 0
0 1 03333333 . 0
0 0 1
0271300 . 0
00333 . 7
19 . 0
3
32
31
L
f
| |{ } { } d
d
B D L 3 . 19
85 . 7
0 1 03333333 . 0
0 0 1
2
1
(
(
(
= | |{ } { }
{ } { }
| |{ } { }
T
x
x
x
D X U
D
d
d B D L
0843 . 70
5617 . 19
85 . 7
0120 . 10 0 0
293333 . 0 00333 . 7 0
2 . 0 1 . 0 3
0843 . 70 5617 . 19 85 . 7
4 . 71
3 . 19
1 0271300 . 0 10000000 . 0
0 1 03333333 . 0
3
2
1
3
2
(
(
(

=
=
)
`
)
`
(
(
110 Rac 2009

{ } { }
T
X 00003 . 7 5 . 2 3
3
=
) )
Rac 56
7. 5 Cr out Decomposi t i on
Component s of t he lower t raingular
mat rix [ L] , l
i,j
are t he same as a
i,j .
Wit h comparable comput at ional
| |
(
(
(
=
13 12 11
a a a
a a a
A
Wit h comparable comput at ional
effort , Crout decomposit ion is
concept ually- simpler t han LU
( Doolit t le) decomposit ion
Wit h similar st rat egy, [ L] and [ U]
mat reices can be compact ed t o [ A]
t o save t he memory usage.
Moreover, since t he original [ A]
mat rix is never used for furt her
| |
| |
(
(
(
(
=
(
(
=
23
13 12
33 32 31
23 22 21
0 0
1 0 0
1 0
1
a
u
u u
U
a a a
a a a A
111 Rac 2009
mat rix is never used for furt her
calculat ion, [ A] can replace [ A] t o
be only one mat rix, again saving
memory
| |
(
(
(
=
33 32 31
22 21
21
0
0 0
a a a
a a
a
L
Numer i cal Rel at i onshi ps
n i a l
i i
,..., 2 , 1 for
1 , 1 ,
= =
n j j i
u l a
u
n , , j,j i u l a l
n j
n j
a
a
u
j
i j
ik ji jk
k j
j
k
kj ik j i j i
j
j
2 1 for
1 for
: ,..., 3 , 2 For
,..., 3 , 2 for
1
1
1
1
, ,
11
1
, 1
+ + =
=
+ = =
=
= =
+ =
=
112 Rac 2009
n , ,j j i
l
u
jj
k j
, 2 1 for
,
+ +
Rac 57
7. 6 Mat r i x I nv er se
| || | | | | | | |
1 1
I A A A A = =

| | { } { } { }
| | { } { } { }
33
23
13
1
3
32
22
12
1
2
31
21
11
1
1
33 23 31
23 22 21
13 12 11
1
0
0
; 1
0
; 0
1
0 1 0
0 0 1
'
'
'
;
'
'
'
;
'
'
'
' ' '
' ' '
' ' '
i i i
I I I I
a
a
a
A
a
a
a
A
a
a
a
A
a a a
a a a
a a a
A
=
(
(
(
=
(
(
(
=

113 Rac 2009
| | { } { } { }
| |{ } { } | |{ } { } | |{ } { }
3
1
3 2
1
2 1
1
1
3 2 1
; ;
1
0 ;
0
1 ;
0
0
1 0 0
0 1 0
i i i i i i
i i i
I A A I A A I A A
I I I I
= = =
)
`
)
`
)
`
=
(
(
=

7. 7 Er r or Anal y si s & Sy st em Condi t i on
Appl i cat i on of I nv er se Mat r i x f or I l l - condi t i on
mat r i x :
S l t h t i [ A] I t t i [ A] t h Scale t he mat rix [ A] ; I nvert mat rix [ A] ; compare t he
order of magnit ude of element s of [ A]
- 1
for I ll- condit ion
mat rix
if not , t hen I ll- condit ion
if not , t hen I ll- condit ion
| || | | |??
1
I A A =
| | | | | |??
1
1
A A =
114 Rac 2009

Rac 58
7. 7 Er r or Anal y si s & Sy st em Condi t i on
Mat r i x Nor m:
Euclidian norms, for vect ors
Frobenius norms, for general mat rices
Uniform vect or norm
Row- sum norm
Mat r i x Condi t i on Number :
| |
1
= A A A Cond
115 Rac 2009
Applicat ion of Cond[ A] Error est imat e of { X} :
| |
A
A
A Cond
X
X A
=
A
7. 7 Speci al Mat r i ces
Special ( Square) Mat rices:
Diagonal syst em
Banded syst em ( e g Tridiagonal) Fig 11 1 Banded syst em ( e. g. Tridiagonal) Fig. 11. 1
They are oft en found in finit e element problems, and requiring
solving wit h a minimum comput at ional effort .
116 Rac 2009
Rac 59
Thomas algorit hm an efficient algorit hm for such mat rices
| |{ } { } | |{ } { }
(
(
(
(
= = '
1 1 12 11
g f e
g f
a a a
a a
R X A B X A
| | | |
`
=
= =
=
(
(
(
(
(
(
(
=
(
(
(
(
(
(
(
=
+
, , 3 , 2
' ;
1 ,
1
11 1
4 4 4
3 3 3
2 2 2
11 11
45 44 43
34 33 32
23 22 21

a g
n k for f e e
a f
f e
g f e
g f e
g f e
A
a a
a a a
a a a
a a a
A
k k k
k k k
n n
Matrix Transformation
117 Rac 2009
=

=
=
)
`
= =
=
+
1 , , 2 , 1
, , 3 , 2
1
1
1 1
n n k for
f
x g r
x
f r x
n k for r e r r
b r
k
k k k
k
n n n
k k k k Forward Substitution
Backward Substitution
Symmet ric Mat rices: a
ij
= a
j i
or, [ A] = [ A]
T
.
Therefore, only requires half t he st orage
118 Rac 2009
Rac 60
7. 8 I t er at i v e Appr oach
Up t o now, t he problem solving met hods for linear
algebraic equat ions are based on det erminist ic approach.
Th ( f d ff d They are accurat e ( apart from round- off error and error
propagat ion) and reasonably efficient for problems wit h
small mat rices
However, for large mat rices, t hey are no longer efficient
Gauss- Seidel and Jocobi met hods use I t erat ive approach,
requiring a set of init ial guesses
119 Rac 2009
The approach is based from t he fundament als of t he linear
algebraic equat ions, wit h init ial guesses: x
1
, x
2
, , x
n
n n
x a x a x a b
x b x a x a x a
1 3 13 2 12 1
... + + +
= = + + +
nn
n n n n n
nn
n
n n n nn n n
n n
n n
n n
a
x a x a x a
a
b
x b x a x a x a
a
x a x a x a
a
b
x b x a x a x a
a a
x b x a x a x a
1 1 , 1 3 2 1 1
2 2 1 1
22
2 3 23 1 21
22
2
2 2 2 2 22 1 21
11 11
1 1 1 2 12 1 11
...
...
...
...
...

+ +
= = + + +
+ +
= = + + +
= = + + +
. .
120 Rac 2009
nn nn
The first Iteration
Ax=b
Rac 61
121 Rac 2009
Gauss-Seidel Jacobi
The difference bet ween Gauss- Seidel and Jacobi is t he first
it erat ion, ( Fig. 11. 4)
St opping crit eria:
j
i
j
i
x x
1
As always in an it erat ive approach, a problem of
divergence may appear.
Convergence Crit eria:
s j
i
i i
i a
x
x x
c c < = % 100
,
=
>
n
j
j i ii
a a
1
,
122 Rac 2009
Some improvement t echniques of Gauss- Seidel is
Relaxat ion:
( )
old
i
ng
i
ns
i
x x x + = 1
=i j
j
Rac 62
Mai n Gauss
i > n
start
a, b, n, x, tol, er
Eliminate
er = 0
i i 1
i = 1
Y
A
Eliminate
j > n
E d
i = i+1
S(i) = ABS [a (i,1) ]
j = j+1
j = 2
er -
1
Substitute
X
Y
N
N
N
Y
N
123 Rac 2009
a (i,j) > s (i)
End
S(i)=ABS[a(i,j)]
A
Y
Subst i t ut e
start
A, n, b, x
X
n
=
i -1
i=i+1
i=n-1
Sum = 0
j n
i = 1+1
Y
N
Y
x
i
= (b
i
sum)/a
i,i
124 Rac 2009
j n
sum =sum+ a (i,j)*x(j)
End
j=j+1
N
Rac 63
8. OPTI MI SATI ON
Essent ially, similar t o root locat ion, opt imisat ion process
seeks for a point on a funct ion.
The difference: The difference:
Root locat ion t o find a root t hat gives t he funct ion zero:
Opt imisat ion t o find a root t hat gives minimum or
maximum value of t he funct ion:
( ) 0 = x f
( ) 0 ' = x f
125 Rac 2009
Why minimising/ maximising??
Minimising weight of mat erial
Minimising cost
Maximising performance

8. OPTI MI SATI ON
Example: Cant ilever Beam
Opt imisat ion Problem:
Design a minimum weight of cant ilever beam ( profile: I -
beam) t hat meet failure and defelect ion crit eria
126 Rac 2010
Design variables : b, h, t
Design parameters :E, , S
y,
L, F
Objective function : .A.L
Rac 64
8. OPTI MI SATI ON
General Component s in Opt imisat ion Problems:
( ) x x x f
n
,..., , : Max/Min
2 1
Objective Function
( )
( )
( )
( ) x x x g
x x x h
x x x h
x x x h
n
n l
n
n
0 ,..., ,
0 ,..., ,
0 ,..., ,
0 ,..., , : Subject to
2 1 1
2 1
2 1 2
2 1 1
.
s
=
=
=
Equality constraints
127 Rac 2010
( )
( )
n i x x x
x x x g
x x x g
u
i i
l
i
n m
n
, , 2 , 1
0 ,..., ,
0 ,..., ,
2 1
2 1 2
.
= s s
s
s
Bound constraints
Inequality constraints
Design variables
8. OPTI MI SATI ON
Cl assi f i cat i on:
The presence of Const raint s:
Const rained Opt imisat ion
Unconst rained Opt imisat ion Unconst rained Opt imisat ion
Number of Design variables:
One- dimensional
Mult i- dimensional
Nat ure of Obj ect ive funct ion & const raint s:
Linear Programming ( if Obj ect ive funct ion and Const raint s
Fig. PT 4.4
128 Rac 2009
Linear Programming ( if Obj ect ive funct ion and Const raint s
are linear)
Quadrat ic Programming ( if Obj ect ive funct ion is quadrat ic
and Const raint s are linear)
Non- linear Programming ( if Obj ect ive funct ion and
Const raint s are nonlinear)
Rac 65
8. OPTI MI SATI ON
8. 2 Appl i cat i on
129 Rac 2009
8. OPTI MI SATI ON
8. 2 Appl i cat i on
130 Rac 2009
Rac 66
8. OPTI MI SATI ON
8. 3 One Di mensi onal
Golden- Sect ion Search
1 5 2 1
l l
Recall: Bisect ion & False- posit ion
Quadrat ic I nt erpolat ion
Curve around t he ext reme is approximat ed by quadrat ic
curve, f
2
( x
0
, x
1
, x
2
, f( x
0
) , f( x
1
) , f( x
2
) )
2
1 5
= d
1
2
2 1
1
l l l
=
+
131 Rac 2009
,
2
(
0
,
1
,
2
, (
0
) , (
1
) , (
2
) )
Find t he maximum of t he f
2
, t o obt ain x
3
.
Repeat t he process wit h new dat apoint , ( x
3
, f( x
3
) )
8. OPTI MI SATI ON
Golden Sect ion Algorit hm:
I nit ial guess, x
u
and x
l
Find golden sect ion dist ance:
( )
d x x
d x x
x x d
u
l
l u
=
+ =
=
2
1
2
1 5
I f f( x
1
) > f( x
2
) x
l new
= x
2
I f f( x
1
) < f( x
2
) x
u new
= x
1
Repeat t he above 3 st eps unt il it reaches convergence ( x
1
x
2
)
132 Rac 2010
Rac 67
8. OPTI MI SATI ON
Quadrat ic I nt erpolat ion:
Take t hree init ial guess
Formulat e quadrat ic equat ion
Find t he maximum/ minimum by different iat e it , result ing
in:
( )( ) ( )( ) ( )( )
( )( ) ( )( ) ( )( )
1 0 2 0 2 1 2 1 0
2
1
2
0 2
2
0
2
2 1
2
2
2
1 0
3
2 2 2 x x x f x x x f x x x f
x x x f x x x f x x x f
x
+ +
+ +
=
133 Rac 2009
8. OPTI MI SATI ON
General Flow Chart
I nit ial guess
Error
check
Select ion
for t he next
evaluat ion
N
Y
Opt imum
134 Rac 2009
Find next
guess
Rac 68
8. OPTI MI SATI ON
Newt on Met hod
Recall Newt on- Rhapson met hod t o obt ain t he root of
t i equat ions
Only, now f( x) = 0, inst ead of f( x) = 0.
Funct ion, f, must be different iable t wice t o find non zero f
( )
( ) x f
x f
x x
i i
"
'
1
=
+
135 Rac 2009
8. OPTI MI SATI ON
8.4 Mul t i di mensi onal , Unconst r ai ned
Most opt imisat ion problems involve mult i- dimensional
Classificat ion:
Non- gradient approach ( Direct Met hod) , e. g. ( Pseudo)
random search, Univariat e & Pat t ern
Gradient Based approach, e. g. Gradient & Hessian,
St eepest Ascent / Descent
136 Rac 2009
Rac 69
8. OPTI MI SATI ON
Pseudo random:
For each design variable, calculat e:
Then: f( x..)
Locat e t he maximum/ minimum
( )r x x x x
l u l
+ =
Where r is a random number
between 0 to 1, generated by
computer
137 Rac 2009
8. OPTI MI SATI ON
Univariat e:
Set I nit ial guess
Perform 1D search while fixing t he ot her variables.
Repeat for each variable, once at a t ime
Repeat he above unt il reaching t he maximum/ minimum
138 Rac 2009
Rac 70
8. OPTI MI SATI ON
Gradient , Hessian:
Direct ional derivat ive or Gradient direct s us t o t he
t t d/ d d ( l ) t d st eepest ascend/ desced ( slope) t owards
maximum/ minimum for each numerical st ep
( )
( ) ( ) ( )
T
n
x
f
x
f
x
f
f
)
`
c
c
c
c
c
c
= V
x x x
x
2 1
139 Rac 2009
8. OPTI MI SATI ON
Gradient , Hessian:
The behaviour of ascending or descending can be checked
i H i t i t t using Hessian mat rix t est :
I f and f has a local minimum
I f and f has a local maximum
|
|
.
|
\
|
c c
c
c
c
c
c
=
y x
f
y
f
x
f
H
.
2
2
2
2
2
0 > H 0
2 2
> c c x f
0 > H
0
2 2
< c c x f
140 Rac 2009
I f f has a saddle point
For evaluat ion of gradient and Hessian, one can use
analyt ical or numerical approach
0 < c c x f
0 < H
Rac 71
8. OPTI MI SATI ON
St eepest Ascend/ Descend:
Proceeding from t he gradient approach, t he complet e st ep by
t i f ll st ep is as follows. .
Set init ial guess
Find t he direct ional gradient
Formulat e one- dimensional direct ional funct ion:
( ) x f V
( )
|
|
.
|
\
|
c
c
+
c
c
+
c
c
+ = h
x
f
x h
x
f
x h
x
f
x f h g
n
n
, , ,
2
2
1
1

141 Rac 2009
Check for t he maximum/ minimum/ saddle point exist ence
Find t he opt imum st ep size, h by different iat ing g( h)
Find t he next st ep of x
i
.
Repeat t he process
8. OPTI MI SATI ON
8.5 Li near , Const r ai ned
Linear Programming
Bound: 1 6 represent limit of solut ion, whilst t he dashed
li t i l f bj t i f t i Z line represent iso- value of obj ect ive funct ion, Z.
142 Rac 2009
Rac 72
8. OPTI MI SATI ON
8.5 Const r ai ned
Possible Cases in Linear Programming
143 Rac 2009
Unique solution Alternate solution
No feasible solution Unbounded problems
8. OPTI MI SATI ON
8.5 Const r ai ned
Graphical Solut ion
Approximate optimum solution:
x
1
= 10; x
2
= 7
144 Rac 2009
Rac 73
8. OPTI MI SATI ON
8.5 Const r ai ned
Simplex Met hod
Based on graphical solut ion, Simplex uses assumpt ion t hat
t h t i l t i li t i t t he opt imum solut ion lies on an ext reme point
Const raint equat ions equalit ies, by int roducing slack
variables
Form t he opt imisat ion int o syst em of linear algebraic
equat ions
Solve it wit h Gauss- Jordan
145 Rac 2009
8. OPTI MI SATI ON
8.5 Const r ai ned
Simplex Met hod for Linear Programming
Obj ect ive funct ion minimizat ion
( ) ( )
j i j i j
b x g x g s s ' 0
( )
k
b x x g = +
*
Obj ect ive funct ion minimizat ion
I nequalit y const raint s
Basic variables ( real design
variables) , x
i.
I nt roducing slack variables, x
k
so
t hat t urning inequalit y const raint s
t o Equalit y const raint s
146 Rac 2009
( )
j k i j
b x x g +
t o Equalit y const raint s
Rac 74
8. OPTI MI SATI ON
Simplex Met hod for Linear Programming ( cot d. )
Then build a t ableu:
V i bl B i i bl l k
n m m
b a a
x x x x x
. . . .

1 0
0 0 1
1 22 12
1 2 1 +
( ) x g
j
*

Variables Basic

Variables Slack
Modified constraints
147 Rac 2010
k
b
( ) x f
*
Modified
objective functions
8. OPTI MI SATI ON
Example of Simplex Met hod
Using Gauss- Jordan, solve t he t ableu int o:
1 2 1 + n m m
x x x x x
*
1
*
*
1 1 , 1
1 2 1
0 0 0
1 0 0
0 0 0 0 1
+
+
+
j
j jn
m
n m m
b
b a
b a
. . . . .
jn
a
Then, solut ion:
wit h t he minimum obj ect ive funct ion of:
148 Rac 2010
* *
2 2
*
1 1
;
m m
b x b x b x = = =
*
1 +
=
j
b f
Rac 75
8. OPTI MI SATI ON
Opt imisat ion Problem:
( ) 5250 900 990 + + f X ( )
( )
( )
( )
0 ,
70 6 3 :
25 3 :
5 . 8 6 . 0 4 . 0 :
5250 900 990 : max
2 1
2 1 3
2 1 2
2 1 1
2 1
>
s +
s
s +
+ +
x x
x x g
x x g
x x g
x x f
X
X
X
X
149 Rac 2010
8. OPTI MI SATI ON
Tableu :
Variables Basic
Variables Slack
70 1 0 0 6 3
25 0 1 0 1 3
5 . 8 0 0 1 6 . 0 4 . 0
5 4 3 2 1
b x x x x x
jn
a

Variables Basic

Variables Slack
150 Rac 2010
5250 0 0 0 900 990
Rac 76
8. OPTI MI SATI ON
Aft er Gauss- Jordan, t he Tableu becomes:
V i bl B i
Variables Slack
4285 . 6 1428 . 0 1428 . 0 0 1 0
4762 . 10 0476 . 0 2857 . 0 0 0 1
4524 . 0 1047 . 0 0285 . 0 1 0 0
5 4 3 2 1

b x x x x x
jn
a
Variables Basic

Variables Slack
Solut ion:
t o give obj c funct ion, f = 21407. 88
151 Rac 2010
88 . 21407 71 . 175 28 . 154 0 0 0
4285 . 6 ; 4762 . 10
2 1
= = x x

MS2200 Anum Part 1

Enviado por

Dados do documento

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

MS2200 Anum Part 1

Enviado por

Direitos autorais:

Formatos disponíveis

Rac 1

(can be adaptable in more advanced Num.

And so on unt il t he last column

110 Rac 2009

114 Rac 2009

Você também pode gostar