Solving Nonlinear Equations

CHAPTER 2

Introduction:

In scientific and engineering studies, a frequently occurring

problem is to find the roots or zeros of equations of the form
f(x) = 0. (Root Finding Problem)

f(x) may be algebraic or transcendental or a combination of

both.

Algebraic functions of the form P(x)= a0 + a1 x + + an xn ,

are called polynomials.

A non-algebraic function is called a transcendental function.

2

Solution Methods

Polynomials up to degree 4 can be solved exactly. Since

finding the root of f(x)=0 is not always possible by
analytical means, we have to go for some other techniques
or methods.

Solution Methods are either Bracketing:

Bisection Method, Regula Falsi Method
or Iterative:
Secant Method, Newton's Method, Graffes Root
Squaring Method,

Rate of Convergence.
Advantages/Disadvantages.
3

Multiplicity

A root p of the equation f(x) = 0 is said to be a Root of

Multiplicity m if f can be written in the form
f ( x) ( x p) m q ( x), where lim q ( x) 0.
x p

A root of multiplicity one is said to be a Simple Root.

Theorem: Let f be a continuous function with m continuous
derivatives. The equation f(x) = 0 has a root of multiplicity m
at x = p if and only if

f ( p) f ( p ) f ( p ) ... f ( m 1) ( p) 0 but f ( m ) ( p) 0.

The Bisection Method

The Bisection method is based on the Intermediate Value
Theorem and Cantor's Intersection Theorem.
These methods essentially work by first finding an interval which is
guaranteed to contain a root and then systematically shrinking the
size of that interval.
Intermediate Value Theorem: Let f(x) be continuous function in
[a,b] and let k be any number between f(a) and f(b). Then, there
exists a number c in (a, b) such that f(c) = k.
Hence, an equation f(x) = 0 where f(x) is a real continuous
function, has at least one root between a and b if f(a) f(b) < 0.
5

Bisection method continued

Cantor's Intersection Theorem:
If a sequence of closed intervals [an, bn] is such that
[an+1, bn+1] [an, bn] for all n, and lim(bn - an)=0 as n ,

then
n 0

, bn

consists of exactly one point.

Algorithm for Bisection Method:

Suppose f(x) is continuous and we have two numbers a1 and b1 such
that f(a1)f(b1)<0, then by Intermediate Value Theorem we know
that there is a root for f(x) between a1 and b1.
6

Bisection method continued

Then divide the interval into two parts and take the middle point
suppose that is c,
if f(c ) =0 then we got the root,
if not so then by IVT the root lies within a and c or c and b,
depending on whether f(a1)f(c) <0 or f(c)f(b1)<0.
Then we have to take the next interval in which the root lies and
continue our process.
Since we are dividing the interval by half each time, we are
approaching to zero, hence this algorithm gives us the approximate
solution. Now we have to put some accuracy criteria to stop the
process.
7

Bisection method continued

Algorithm for Bisection Method:
Step 1: Choose a and b as two initial guesses for the root such
that f(a) f(b) < 0.
Step 2: Set x= (a+b)/2.
Step 3: If f(a) f(x) < 0, the root lies in the interval (a, x).
Then, set b = x
else set a = x
end if (|a b| 1 or f(x) 2)
Remark: The method may produce a false root if f(x) is
8
discontinuous on [a, b].

Bisection method continued

For the given values of 1 and 2, the iteration continues until

|bn - an| < 1 and/or |f(x)| < 2.
The root correct up to desired accuracy is (an - bn)/2.
Advantage: This method definitely converges to the root of any
type of non-linear equation.
Disadvantages:
This method is too slow (linear) i.e.

en 1 c en ,

where c <1.

We need two initial guesses for this method.

9

Example 1: Find a real root of the equation x3 - 2x - 5 = 0,

correct up-to 3-significant digits.

10

Bisection method continued

Example 1:

Find a real root of the equation x3-2x-5 = 0, correct up-to 3-significant

digits.
Solution: Let f(x)=x3-2x-5. Now f(2)=-1 and f(3)=16. Hence, by IVT,
there exist a root between 2 and 3.
Iteration 1. Now we set a0=2, b0=3, and hence x= (a0+b0)/2=2.5.
Since f(2.5)= 5.625, we have f(2) f(2.5) < 0, set a1=2, b1 = 2.5 for the
next iteration.
Iteration 2. a1=2, b1=2.5, x=(a1+b1)/2= 2.25, f(2.25)= 1.890625.
Since f(2)f(2.25)<0, set a2=2, b2 = 2.25.

11

Bisection method continued Example

1:

Iteration 3. a2=2, b2=2.25, x=(a2+b2)/2= 2.125, f(2.125)=0.3457.

Since f(2)f(2.125)<0, set a3=2, b3=2.125.
Iteration 4. a3=2, b3=2.125, x=(a3+b3)/2= 2.0625, f(2.0625)=0.3513. Since f(2)f(2.0625)>0, set a4=2.0625, b4=2.125.
Iteration 5. a4=2.0625,
f(2.09375)=-0.0089.

b4=2.125,

x=(a4+b4)/2

=2.09375,

Since f(2.0625)f(2.09375)>0, set a5=2.09375, b5=2.125.

12

Bisection method continued Example

1:

Iteration 6. a5=2.09375, b5=2.125 Now, x=(a5+b5)/2=2.10938,

f(2.10938)=0.1668. Since f(2.09375)f(2.10938)>0, set a6=2.09375,
b6=2.10938.
Iteration 7. a6=2.09375, b6=2.10938, x=(a6+b6)/2 =2.10156.
Thus, a 3-significant digits root of the given equation is 2.10156.
Further iterations can be made to get better accuracy.

13

Example 1
k
1
2
3
4
5
6
7
8

a
2.00000
2.00000
2.00000
2.00000
2.06250
2.09375
2.09375
2.09375

b
3.00000
2.50000
2.25000
2.12500
2.12500
2.12500
2.10938
2.10156

f(a)
-1.00000
-1.00000
-1.00000
-1.00000
-0.35132
-0.00894
-0.00894
-0.00894

f(b)
16.00000
5.62500
1.89063
0.34570
0.34570
0.34570
0.16684
0.07856

x
f(x)
2.50000 5.62500
2.25000 1.89063
2.12500 0.34570
2.06250 -0.35132
2.09375 -0.00894
2.10938 0.16684
2.10156 0.07856
2.09766 0.03471

14

Bisection method continued Exercises:

Problem1: Perform 5 Iterations of the bisection method to obtain

the smallest positive root of the equation
x3-5x+1 =0.
Problem2: Determine the smallest positive root correct up to two
decimal places using bisection method for
tan x + tanh x=0.

15

16

The Secant Method

Let x0 and x1 be the two approximation of the root . If the function f
is linear, the straight line passes through (x0, f(x0)) and (x1, f(x1))
would intersect the X-axis at the root (y=0). But since f is not linear,
the intersection of the line with X-axis is not at x= but this should
be close to it.
The equation of chord through (x0, f(x0)) and (x1,f(x1)) is given by

f ( x1 ) f ( x0 )
y f ( x0 )
( x x 0 ).
x1 x0

17

secant method continued

At the intersection with X-axis, y=0. So,

f ( x0 )( x1 x0 )
x x0
,
f ( x1 ) f ( x0 )
On simplification we get

f ( x0 )( x1 x0 ) x0 f ( x1 ) x1 f ( x0 )
x x0

,
f ( x1 ) f ( x0 )
f ( x1 ) f ( x0 )
Now we take this x as our
approximation and denote it as x2

next
18

secant method continued

From x1 and x2 we can calculate our next
approximation x3 using the previous technique
and in general, we can write the iteration
scheme as

x n 1

x n 1 f ( x n ) x n f ( x n 1 )

, n 1,2,3,
f ( x n ) f ( x n 1 )

We should start with x0 and x1 such that x1 is

closer to the root than x0 to get better result.
19

secant method continued

Geometrical Interpretation of Secant Method

The approximation points are a,b, x1,x2,x3,20

secant method continued Example:

Find the root of f(x)=x3+x2-3x-3 correct up to 3significant digits with initial approximation
x0=1 and x1=2 by secant method.
Solution: The iteration scheme of the secant
method is, x n 1 f ( x n ) x n f ( x n 1 )

x n 1

f ( x n ) f ( x n 1 )

, n 1,2,3,

Using this formula we get

x0 f ( x1 ) x1 f ( x0 ) 1 f (2) 2 f (1)
x2

1.571429,
f ( x1 ) f ( x 0 )
f (2) f (1)
21

secant method continued

Applying the formula again we get x3 as

x1 f ( x 2 ) x 2 f ( x1 ) 2 f (1.571419) 1.571419 f (2)
x3

1.705411,
f ( x 2 ) f ( x1 )
f (2) f (1.571419)

Similarly computing further we get

x 2 f ( x3 ) x3 f ( x 2 )
x4
1.735136,
f ( x3 ) f ( x 2 )
x3 f ( x 4 ) x 4 f ( x3 )
x5
1.731996,
f ( x 4 ) f ( x3 )

Thus the root correct up to 3-significant digits

of the given equation by secant method is
22
1.731996.

secant method continued

Advantages and Disadvantages

Advantages:
Converges fast (if it converges)
comparison with Bisection method.

in

Requires two guesses that do not need to

bracket the root.
Only requires one function evaluation per
iteration.
Avoids calculation of derivatives.
23

secant method continued

Advantages and Disadvantages

Disadvantages:
Division by zero
In this method, it is not necessary to have
two initial guesses bracketing the root, but
on the other hand, convergence is not
guaranteed.
In some cases, swapping the two initial
guesses x0 and x1 will change the behavior of
the method from convergent to divergent.
24

secant method continued Order

of Convergence:

Let xn be a sequence of approximations to the

root by a given method. Let en = - xn be the
error in the n-th approximation of xn.
If there exists a number c 0 (independent of
n) such that
|e |

lim | e
n

n 1

then p is the order of convergence of the

method.
25

secant method continued

Order of Convergence of Secant Method:

Prove that the order of convergence of secant

1 5

method is approximately 1.618.

Proof: We know that by Secant Method the

sequence is generated by

x n 1

x n 1 f ( x n ) x n f ( x n 1 )

, n 1,2,3,
f ( x n ) f ( x n 1 )

Further let the sequence {x0, x1,

converges to the root, (i.e. f()=0).

x2,

}
26

secant method continued

Order of Convergence of Secant Method:

Now we have en+1= - xn+1

x n 1 f ( x n ) x n f ( x n 1 )

,
f ( x n ) f ( x n 1 )
Thus

en 1

en xn , or xn en , xn1 en1 .

( en 1 ) f ( en ) ( en ) f ( en 1 )

. (1
f ( en ) f ( en 1 )
)
27

secant method continued

Order of Convergence of Secant Method:

Expanding the functions by Taylors series we
2
2
get
en
en
f ( en ) f ( ) en f ( )

2!

f ( ) en f ( )
2

en 1
f ( en 1 ) f ( ) en 1 f ( )
f ( )
2!
2
en 1
en 1 f ( )
f ( )
2!

2!

f ( )

Substituting these values in (1) and

neglecting the higher degree terms and
28
simplifying

secant method continued

Order of Convergence of Secant Method:

we get
1
f ( )
en 1 en 1en
.
(2)
2
f ( )
Let

1 f ( )

.
2 f ( )

(A constant)

p
e

e
e
.
n 1 n Suppose e n 1 e n
Then we have n 1
p . On substituting these values
then
en en 1

in (2) we get

en 1

en
en

1/ p

en 1

(1 / p ) 1
1 / p en
.

29

secant method continued

Order of Convergence of Secant Method:

Comparing

(1 / p ) 1
e

e
en 1 en and
n 1
n
1/ p

and equating the powers of en on right sides we

get
(1 / p ) 1 p or p 2 p 1 0
p

Solving for p we get

1 5
1.618
2

Hence the order of convergence is 1.618.

Therefore the secant method is faster than
30
linear and slower than Newton's quadratic

Regula Falsi Method:

When the bracketing is done for the secant
method, it is known as linear interpolation or
the method of false position (in Latin, Regula
Falsi).
This technique for Regula Falsi method is
similar to bisection except the next iterate is
taken at the intersection of a line between the
pair of X-values and X-axis rather than at the
midpoint.
This method gives faster convergence in
31
comparison with bisection method, but at the

Regula Falsi Method continued

Algorithm for Regula Falsi Method:
Step 1: Choose x0 and x1 as two guesses for the
root such that f(x0 ) f(x1)<0.
x0 f ( x1 ) x1 f ( x0 )
Step 2: Set x 2 f ( x ) f ( x )
1
0
Step 3:

If f(x0)f(x2)<0, the root lies in the interval (x0, x2).

Then, set x1=x2 and go to Step 2.
If f(x0)f(x2)>0, the root lies in the interval (x0, x2).
Then, set x0=x2 and go to Step 2.
32

Regula Falsi Method continued

Algorithm for Regula Falsi Method:

If f(x0)f(x2)=0, x2 is a root of the equation and the

computation may be terminated.

Continue till a root is obtained to desired

accuracy

33

Regula Falsi Method continued

General remarks on Regula Falsi Method:

Asymptotically, the other end-point will remain

fixed for all subsequent iterations while one
end-point always being updated.

As a result, unlike the bisection method, the

width of the bracket does not tend to zero. As a
consequence, Regula Falsi approximation to f(x),
which is used to pick the false position, does not
improve in its quality.

If the initial end-points x0 and x1 are chosen such

that f(x0) and f(x1) are of opposite signs, then one
of the end-points will converge to a root of f. 34

Regula Falsi method continued

Example:

Let f(x)=2x3-4x2+3x. Let x0=-1 and x1=1. The left

end, -1, is never replaced and thus the width of
the bracket never falls below 1. Hence, the
right endpoint approaches 0 at a linear rate
(with a speed of convergence of 2/3).
To accelerate the rate of convergence in such
cases, the Modified Regula Falsi Method is
used. In this modified rule, Set
x0 f ( x1 ) x1 f ( x0 ) / 2
x2
f ( x1 ) f ( x0 ) / 2
if successive x2 falls on right of .
35

Regula Falsi method continued Example:

or Set

x0 f ( x1 ) / 2 x1 f ( x0 )
x2
f ( x1 ) / 2 f ( x0 )

if successive x2 falls on left of .

36

Regula Falsi method continued Example:

Find a real root of the equation x3-2x-5=0 correct

up-to 3-significant digits by using Regula
Falsi method.
Solution: Here, f(2)=-1 and f(3)=16 and hence a
root lies in (2,3). Let x0=2 and x1=3. We get x2 as
x0 f ( x1 ) x1 f ( x0 )
x2
2.058823592
f ( x1 ) f ( x0 )

Now, f(x2)=-0.390799917 and hence the root lies

between 2.058823529 and 3.
37

Regula Falsi method continued Example:

Computing x3 we get

x1 f ( x 2 ) x 2 f ( x1 )
x3
2.08126366
f ( x 2 ) f ( x1 )
Now, f(x3)=-0.147204057 and hence the root lies
between 2.08126366 and 3. Similarly

x 2 f ( x3 ) x3 f ( x 2 )
x4
2.089639211
f ( x3 ) f ( x 2 )

Thus, the root correct up to 3-significant digits

of the given equation by secant method 38is

Newton's Method:

Choose a value that is reasonably close to

the root of the equation.

Replace the function by its tangent.

Computes the zero of this tangent.

The zero of this tangent will be a better

approximation to root of the equation and
the procedure can be iterated.

In Newton's Method, we replace the

chord in the secant method by a
39
tangent.

Geometrical Interpretation of Newtons Method

40

Newton's Method ContinuedTheorem:

Newton-Raphson

Theorem: Assume

that fC2[a,b] and there exists a number

[a,b], where f()=0. If f() 0, then there

exists a >0 such

the sequence
{xn}nthat
0
defined by the iteration
f ( xn )
x n 1 x n
, for n 0,1,2,3,
f ( x n )
will convergex to[for
any
initial
approximation

]
0
.
41

Newton-Raphson Theorem Continued

Proof: Since fC2[a,b], using Taylor's series
expansion of f around x0, we have
( x x0 ) 2
f ( x) f ( x0 ) ( x x0 ) f ( x 0 )
f (c).
2!

where c lies in between x and x0.

Substituting x= in the Taylor's series with the
fact f()=0, we get
( x0 ) 2
0 f ( x0 ) ( x0 ) f ( x 0 )
f (c).
2!

42

Newton-Raphson Theorem Continued

If x0 is close enough to , then the last term on
the right side will be small compared to the
sum of the first two terms. Hence it can be
neglected and we can use the approximation,

0 f ( x0 ) ( x0 ) f ( x0 ).
Solving for we get

x0 f ( x0 ) / f ( x0 ).

43

Newton-Raphson Theorem Continued

This is used to define the next approximation x1
to the root as

f ( x0 )
x1 x0
.
f ( x0 )
In general the Newton Method scheme is
given by

x n 1

f ( xn )
xn
, for n 0,1,2,3,
f ( x n )

44

Newton's method continued Example:

Find a real root of the equation f(x)=3x+sin x- ex

correct up-to 3-significant digits by Newton's
method.
Solution: Here, f(x)=3x+sin x-exC2(). Hence,
f'(x)=3+cos x-ex. The iterative scheme for
Newton's Method is

x n 1

f ( xn )
3 x n sin x n e
xn
xn
. n 0,1,2,
xn
f ( x n )
3 cos x n e
xn

Let x0=0. Thus, we have x1=0-(-1/3) = 0.33333

45

Newton-Raphson Method

46

Convergence of Newton's Method:

In Newton's Method, fC2[a,b], f()=0 and f'()0,
where [a,b]. Hence,

f ( xn )
f ( xn )
en 1 xn 1 xn
en
f ( xn )
f ( xn )

2
f ( en )
f ( ) en f ( ) (en / 2) f ( n )
en
en
.
f ( en )
f ( ) en f ( n )

Where n, n lies between and xn .

47

Convergence of Newton's Method:

2

en f ( ) (en / 2) f ( n )
en
.
en f ( n )

f ( ) 1
f ( )

f ( n )

en en (en / 2)
f ( )

en f ( n )
2
1
O(en ) .
f ( )

(by using the Binomial expansion)

2

en 1

f ( n ) f ( n ) O(en ).
f ( ) 2

48

Convergence of Newton's Method:

As n, n and . Thus

| en 1 | | f ( ) |
lim

c 0.
2
n | e |
2 | f ( ) |
n
Thus,
Newton's
method
converges
quadratically i.e. Order of convergence p=2.

49

Advantages of Newton method

It is rapidly convergent in most cases since
p=2.
It is simple in its formulation, and therefore
relatively easy to apply and program.
It is intuitive in its construction. This means
it is easier to understand its behavior, when
it is likely to behave well and when it may
behave poorly.
50

Disadvantages of Newton method:

It may not converge.
It is likely to have difficulty if f'(xn)=0. That
means the X-axis is tangent to the graph of
y=f(x) at x=xn.
The closer the derivative goes to zero, the
worse
Newton's Method behaves. Flat
tangents may diverge the scheme.
This method can spin forever if there is no real
root of the equation. (Example: x2 + 2 = 0).
It needs to know both f(x) and f'(x). Contrast this
51

Newton's Method for Multiple Roots:

A root is called a multiple root of order
m, (m) if f(x) = (x- )m h(x) such that h() 0.
For m=1, is called simple root of f(x).
The Newton's method
x n 1 x n

f ( xn )

f ( x n )

converges quadratically, when is a simple

root of f(x)=0, but it converges linearly when
is a multiple root of f(x) = 0.
52

Fixed Point Method (FPM) Continued

Modified Newton's Method:

In the modified
method, we take

x n 1

(accelerated)

Newton's

f ( xn )
xn m
,
f ( x n )

if is a multiple root of order m of f(x)=0 and it

converges quadratically.

53

Example:
Find the approximate value of the double
root of x3 -2x2 - 0.75 x + 2.25 =0, that is correct
up to 3 decimal places and close to 1 such
that iterations converges quadratically. Take
x0 =1.
Solution: Since the root of the equation is a
double root, we will use modified Newton's
method
in
order
to
get
quadratic
convergence.
54

The End of Chapter 2

55