Escolar Documentos
Profissional Documentos
Cultura Documentos
Lecture 02:
Lecture 02
Solution of Nonlinear Equations
( Root
Finding Problems )
Definitions
Classification of Methods
Analytical Solutions
Graphical Methods
Numerical Methods
Bracketing Methods
Open Methods
Convergence Notations
Reading Assignment:
Roots of Equations
A number r that satisfies an equation is called a
root of the equation.
The equation : x 4 3 x 3 7 x 2 15 x 18
has four roots : 2, 3, 3 , and 1 .
i.e., x 4 3 x 3 7 x 2 15 x 18 ( x 2)( x 3) 2 ( x 1)
Zeros of a Function
Let f(x) be a real-valued function of a real
variable. Any number r for which f(r)=0 is
called a zero of the function.
Examples:
2 and 3 are zeros of the function f(x) = (x-2)(x-3).
f(x)
Simple Zeros
f ( x) x 1( x 2)
f ( x) ( x 1) x 2 x x 2
2
Multiple Zeros
f ( x) x 1
f ( x) x 1 x 2 x 1
has double zeros (zero with muliplicity 2) at x 1
2
Multiple Zeros
f ( x) x 3
f ( x) x
has a zero with muliplicity 3 at x 0
3
Facts
Any nth order polynomial has exactly n
zeros (counting real and complex zeros
with their multiplicities).
Any polynomial with an odd order has at
least one real zero.
If a function has a zero at x=r with
multiplicity m then the function and its
first (m-1) derivatives are zero at x=r
and the mth derivative at r is not zero.
10
Solution Methods
Several ways to solve nonlinear equations are
possible:
Analytical Solutions
Graphical Solutions
Numerical Solutions
Open methods
Bracketing methods
12
Analytical Methods
Analytical Solutions are available for special
equations only.
Analytical solution of : a x 2 b x c 0
b b 2 4ac
roots
2a
Graphical Methods
Solve
xe
Root
14
Numerical Methods
Many methods are available to solve nonlinear
equations:
Bisection Method
Newtons Method
These will be covered
in ECEG-2112
Secant Method
False position Method
Mullers Method
Bairstows Method
Fixed point iterations
.
15
Bracketing Methods
Bisection method
False position method
16
Open Methods
In the open methods, the method starts
with one or more initial guess points. In
each iteration, a new guess of the root is
obtained.
Open methods are usually more efficient
than bracketing methods.
They may not converge to a root.
17
Convergence Notation
A sequence x1 , x2 ,..., xn ,... is said to converge to x if
to every 0 there exists N such that :
xn x n N
18
Convergence Notation
Let x1 , x2 ,...., converge to x.
Linear Convergence :
Quadratic Convergence :
Convergence of order P :
xn 1 x
xn x
xn 1 x
xn x
xn 1 x
xn x
C
C
C
19
Speed of Convergence
We can compare different methods in
terms of their convergence rate.
Quadratic convergence is faster than
linear convergence.
A method with convergence order q
converges faster than a method with
convergence order p if q>p.
Methods of convergence order p>1 are
said to have super linear convergence.
20
Bisection Method
21
Introduction
f(a)
b
f(b)
23
Examples
Bisection method
can be used to find
one of the zeros.
25
Bisection Method
Bisection Method
Assumptions:
Given an interval [a,b]
f(x) is continuous on [a,b]
f(a) and f(b) have opposite signs.
These assumptions ensure the existence of at
least one zero in the interval [a,b] and the
bisection method can be used to obtain a smaller
interval that contains the zero.
27
Bisection Algorithm
Assumptions:
f(x) is continuous on [a,b]
f(a) f(b) < 0
Algorithm:
Loop
1. Compute the mid point c=(a+b)/2
2. Evaluate f(c)
3. If f(a) f(c) < 0 then new interval [a, c]
If f(a) f(c) > 0 then new interval [c, b]
End loop
f(a)
c b
a
f(b)
28
Bisection Method
b0
a0
a1
a2
29
Example
+
30
no
yes
is
yes
b=c; v= w
no
is
(b-a) /2<
Stop
u w <0
a=c; u= w
31
Example
Can you use Bisection method to find a zero of :
f ( x) x 3 3 x 1 in the interval [0,2]?
Answer:
f ( x) is continuous on [0,2]
and f(0) * f(2) (1)(3) 3 0
Assumptions are not satisfied
Bisection method can not be used
32
Example
Can you use Bisection method to find a zero of :
f ( x) x 3 3 x 1 in the interval [0,1]?
Answer:
f ( x) is continuous on [0,1]
and f(0) * f(1) (1)(-1) 1 0
Assumptions are satisfied
Bisection method can be used
33
34
35
Stopping Criteria
Two common stopping criteria
1.
2.
36
Stopping Criteria
cn :
r:
After n iterations :
0
b
x
error r - cn Ean n n
2
2
37
Convergence Analysis
Given f ( x), a, b, and
How many iterations are needed such that : x - r
where r is the zero of f(x) and x is the
bisection estimate (i.e., x ck ) ?
log(b a ) log( )
n
log(2)
38
Form
log(b a ) log( )
n
log(2)
width of initial interval
ba
n log 2
log 2
desired error
39
Example
a 6, b 7, 0.0005
How many iterations are needed such that : x - r ?
log(b a ) log( ) log(1) log(0.0005)
n
10.9658
log(2)
log(2)
n 11
40
Example
Question
Question
Question
Question
1:
2:
3:
4:
What is f (x) ?
Are the assumptions satisfied ?
How many iterations are needed ?
How to compute the new estimate ?
41
42
Bisection Method
Initial Interval
f(a)=-0.3776
a =0.5
f(b) =0.2784
c= 0.7
b= 0.9
43
Bisection Method
-0.3776
0.5
-0.0648
0.7
-0.0648
0.7
0.1033
0.8
0.2784
0.9
0.2784
0.9
44
Bisection Method
-0.0648
0.7
-0.0648
0.70
0.0183
0.1033
0.75
0.8
-0.0235
0.0183
0.725
0.75
45
Summary
After 5 iterations:
46
c=
0.7000
fc =
-0.0648
c=
0.8000
fc =
0.1033
c=
0.7500
fc =
0.0183
c=
0.7250
fc =
-0.0235
47
Example
Find the root of:
48
Example
Iteration
c= (a+b)
2
f(c)
(b-a)
2
0.5
-0.375
0.5
0.5
0.25
0.266
0.25
0.25
0.5
.375
-7.23E-3
0.125
0.25
0.375
0.3125
9.30E-2
0.0625
0.3125
0.375
0.34375
9.37E-3
0.03125
49
Bisection Method
Advantages
Simple and easy to implement
One function evaluation per iteration
The size of the interval containing the zero is reduced
by 50% after each iteration
The number of iterations can be determined a priori
No knowledge of the derivative is needed
The function does not have to be differentiable
Disadvantage
Slow to converge
Good intermediate approximations may be discarded
50
Newton-Raphson Method
Assumptions
Interpretation
Examples
Convergence Analysis
51
Newton-Raphson Method
(Also known as Newtons Method)
Assumptions:
52
53
f ( xi )
A new guess of the root : xi 1 xi
f ' ( xi )
54
Newtons Method
Given f ( x ), f ' ( x ), x0
F ( X ) X * *3 3 * X * *2 1
Assumputio n f ' ( x0 ) 0
FP ( X ) 3 * X * *2 6 * X
X 4
__________ __________ __
for i 0: n
xi 1 xi
end
FORTRAN PROGRAM
DO 10 I 1, 5
f ( xi )
f ' ( xi )
10
X X F ( X ) / FP ( X )
PRINT *, X
CONTINUE
STOP
END
55
Newtons Method
Given f ( x ), f ' ( x ), x0
F.m
Assumputio n f ' ( x0 ) 0
__________ __________ __
for i 0 : n
xi 1 xi
end
F X ^3 3 * X ^ 2 1
function [ FP] FP( X )
FP.m
f ( xi )
f ' ( xi )
function [ F ] F ( X )
FP 3 * X ^ 2 6 * X
% MATLAB PROGRAM
X 4
for i 1 : 5
X X F ( X ) / FP ( X )
end
56
Example
Find a zero of the function f(x) x 3 2 x 2 x 3 , x0 4
f ' (x) 3 x 2 4 x 1
Iteration 1 :
f ( x0 )
33
x1 x0
4
3
f ' ( x0 )
33
Iteration 2 :
f ( x1 )
9
x2 x1
3 2.4375
f ' ( x1 )
16
Iteration 3 :
f ( x2 )
2.0369
x3 x2
2.4375
2.2130
f ' ( x2 )
9.0742
57
Example
k (Iteration)
xk
f(xk)
f(xk)
xk+1
|xk+1 xk|
33
33
16
2.4375
0.5625
2.4375
2.0369
9.0742
2.2130
0.2245
2.2130
0.2564
6.8404
2.1756
0.0384
2.1756
0.0065
6.4969
2.1746
0.0010
58
Convergence Analysis
Theorem :
Let f(x), f ' (x) and f ' ' (x) be continuous at x r
where f(r) 0. If f ' (r) 0 then there exists 0
such that x0 -r
1
C
2
xk 1-r
xk -r
min f ' ( x )
x0 -r
59
Convergence Analysis
Remarks
60
Multiple Roots
f ( x) x 3
f ( x) x 1
x0
x1
x0
x1=x3=x5
x0=x2=x4
f1 ( x1 , x2 ,...)
F ( X ) f 2 ( x1 , x2 ,...) ,
f1
x
1
f 2
F '(X )
x1
f1
x2
f 2
x2
66
Example
y x 2 0.5 x
1
2x 1
1
F 2
, X0
, F '
x 5 xy y
2 x 5 y 5 x 1
0
67
,
F
'
1
2
x
5
y
5
x
1
2
6
x
5
xy
1 1 1
X1
0
2
Iteration 2 :
0.5 1.25
1 0.25
1
0.0625
1.5
F
, F ' 1.25 7.25
0.25
1
1.25 1.5
X2
0.0625 1.2332
- 0.25 0.2126
68
Example
Try this
1
y x 2 1 x
2 x 1
0
F 2
, X0
, F '
2
4 y 1
2x
0
x 2y y
69
Example
Solution
Iteration
_____________________________________________________________
Xk
0
0
1
0
0.6
0.2
0.5287
0.1969
0.5257
0.1980
0.5257
0.1980
70
Secant Method
Secant Method
Examples
Convergence Analysis
71
Secant Method
f ( x h) f ( x )
f ' ( x)
h
if xi and xi 1 are two initial points :
f ( xi ) f ( xi 1 )
f ' ( xi )
( xi xi 1 )
f ( xi )
xi 1 xi
f ( xi ) f ( xi 1 )
( xi xi 1 )
( xi xi 1 )
xi f ( xi )
f ( xi ) f ( xi 1 )
73
Secant Method
Assumptions :
Two initial points xi and xi 1
such that f ( xi ) f ( xi 1 )
New estimate (Secant Method) :
( xi xi 1 )
xi 1 xi f ( xi )
f ( xi ) f ( xi 1 )
74
Secant Method
f ( x) x 2 x 0.5
2
x0 0
x1 1
( xi xi 1 )
xi 1 xi f ( xi )
f ( xi ) f ( xi 1 )
75
Yes
xi 1 xi
Stop
76
f ( xi xi ) f ( xi )
xi f ( xi )
xi
f ( xi xi ) f ( xi )
xi
Problem : How to select ?
If not selected properly, the method may diverge .
77
Example
50
40
30
20
10
0
-10
-20
-30
-40
-2
-1.5
-1
-0.5
0.5
1.5
78
Example
x(i)
f(x(i))
x(i+1)
|x(i+1)-x(i)|
-1.0000
1.0000
-1.1000
0.1000
-1.1000
0.0585
-1.1062
0. 0062
-1.1062
0.0102
-1.1052
0.0009
-1.1052
0.0001
-1.1052
0.0000
79
Convergence Analysis
xi 1 r
xi r
r : root
C,
1.62
80
Comparison of Root
Finding Methods
Advantages/disadvantages
Examples
81
Summary
Method
Pros
Cons
Bisection
- Slow
- Needs an interval [a,b]
containing the root, i.e.,
f(a)f(b)<0
Newton
- May diverge
- Needs derivative and an
initial guess x0 such that
f(x0) is nonzero
Secant
- May diverge
- Needs two initial points
guess x0, x1 such that
f(x0)- f(x1) is nonzero
82
Example
Use Secant method to find the root of :
f ( x) x 6 x 1
Two initial points x0 1 and x1 1.5
( xi xi 1 )
xi 1 xi f ( xi )
f ( xi ) f ( xi 1 )
83
Solution
_______________________________
k
xk
f(xk)
_______________________________
0
1.0000 -1.0000
1
1.5000 8.8906
2
1.0506 -0.7062
3
1.0836 -0.4645
4
1.1472 0.1321
5
1.1331 -0.0165
6
1.1347 -0.0005
84
Example
Use Newton' s Method to find a root of :
f ( x) x 3 x 1
Use the initial point : x0 1.
Stop after three iterations, or
if xk 1 xk 0.001, or
if
f ( xk ) 0.0001.
85
k
xk
f(xk)
f(xk)
ERROR
______________________________________
0
1.0000 -1.0000 2.0000
1
1.5000 0.8750 5.7500 0.1522
2
1.3478 0.1007 4.4499 0.0226
3
1.3252 0.0021 4.2685 0.0005
4
1.3247 0.0000 4.2646 0.0000
5
1.3247 0.0000 4.2646 0.0000
86
Example
Use Newton' s Method to find a root of :
f ( x) e x x
Use the initial point : x0 1.
Stop after three iterations, or
if xk 1 xk 0.001, or
if
f ( xk ) 0.0001.
87
Example
Use Newton' s Method to find a root of :
f ( x ) e x x,
xk
f ( xk )
f ' ( x ) e x 1
f ' ( xk )
f ( xk )
f ' ( xk )
88
Example
Estimates of the root of:
0.60000000000000
0.74401731944598
0.73909047688624
0.73908513322147
0.73908513321516
1
4
10
14
x-cos(x)=0.
Initial guess
correct digit
correct digits
correct digits
correct digits
89
Example
In estimating the root of: x-cos(x)=0, to
get more than 13 correct digits:
4 iterations of Newton (x0=0.8)
43 iterations of Bisection method (initial
interval [0.6, 0.8])
5 iterations of Secant method
( x0=0.6, x1=0.8)
90