Numerische Methoden Lecture Notes

Mathematical Methods
Martin Stmpfle
January 30, 2013
Contents
1 Numerical Analysis
2 Ordinary Differential Equations
2.1 Introduction . . . . . . . . . . . . . . . .
2.1.1 Theoretical Basics . . . . . . . .
2.1.2 Numerical Basics . . . . . . . . .
2.2 Eulers Method and Variants . . . . . . .
2.2.1 Eulers Method . . . . . . . . . .
2.2.2 Implicit Eulers Method . . . . .
2.2.3 Improved Eulers Method . . . .
2.2.4 Heuns Method . . . . . . . . . .
2.3 Runge-Kutta Methods . . . . . . . . . .
2.3.1 Runge-Kutta Methods . . . . . .
2.3.2 Step Size Control . . . . . . . . .
2.3.3 The One-Step Approach . . . . .
2.4 Multistep Methods . . . . . . . . . . . .
2.4.1 Linear Multistep Methods . . . .
2.4.2 Adams-Bashforth Methods . . .
2.4.3 Adams-Moulton Methods . . . .
2.4.4 Predictor-Corrector Methods . .
2.4.5 The General Multistep Approach
2.5 Stability and Stiffness . . . . . . . . . . .
2.5.1 Stability of One-Step Methods .
2.5.2 Stability of Multistep Methods .
2.5.3 Stiffness . . . . . . . . . . . . . .
2.6 Initial Value Problems . . . . . . . . . . .
2.6.1 Summary . . . . . . . . . . . . .
2.6.2 Matlab Functions . . . . . . . . .
2.7 Boundary Value Problems . . . . . . . .
2.7.1 Shooting Methods . . . . . . . .
2.7.2 Finite Difference Methods . . . .
2.8 Applications . . . . . . . . . . . . . . . .
2.8.1 Biology . . . . . . . . . . . . . . .
2.8.2 Mechanics . . . . . . . . . . . . .
2.8.3 Engineering . . . . . . . . . . . .
2.8.4 Vehicle Dynamics . . . . . . . . .
2.9 Exercises . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
7
8
8
17
19
19
20
21
22
23
23
26
27
28
28
29
30
31
32
33
33
39
41
44
44
44
45
45
45
47
47
49
50
51
53
4
3 Partial Differential Equations
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.1 Theoretical Basics . . . . . . . . . . . . . . . . . .
3.1.2 Numerical Basics . . . . . . . . . . . . . . . . . . .
3.2 Finite Difference Methods . . . . . . . . . . . . . . . . . .
3.2.1 Finite Differences . . . . . . . . . . . . . . . . . . .
3.2.2 Difference Methods for 2D Elliptic Equations . . .
3.2.3 Difference Methods for 1D Parabolic Equations .
3.2.4 Difference Methods for 2D Parabolic Equations .
3.2.5 Difference Methods for 1D Hyperbolic Equations .
3.3 Finite Element Methods . . . . . . . . . . . . . . . . . . .
3.3.1 Meshes, Partitions, and Triangulations . . . . . . .
3.3.2 Variational Problems . . . . . . . . . . . . . . . . .
3.3.3 Function Spaces . . . . . . . . . . . . . . . . . . .
3.3.4 Piecewise Linear Finite Elements . . . . . . . . . .
3.3.5 Galerkins Method . . . . . . . . . . . . . . . . . .
3.3.6 Rayleigh-Ritzs Method . . . . . . . . . . . . . . .
3.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.1 Heat Distribution . . . . . . . . . . . . . . . . . . .
3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
57
58
58
65
67
67
69
74
81
84
87
87
90
97
97
98
99
114
114
116
A Appendix
121
A.1 Greek Letters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Bibliography
123
Index
125
1 Numerical Analysis
2 Ordinary Differential Equations
Ordinary differential equations are equations where the unknown is a funtion in one variable. Many models from engineering sciences include ordinary differential equations. The
most popular class are multi body systems, where several components are linked together
by rods, springs, dampers, or elastomers. Depending on the constraints we distinguish
between initial and boundary value problems.
2.1 Introduction
We briefly discuss the basics of ordinary differential equations. Besides theoretical aspects
we introduce numerical definitions as well.
2.1.1 Theoretical Basics

Ordinary differential equations can be classified according to their properties. Among
these characteristics are form, order, linearity, or the type of given boundary values.
Definition 2.1 (Ordinary differential equation)

An equation involving a function y(t) of one independent variable t and its n derivatives
y (t), . . . , y (n) (t) is called an ordinary differential equation (ODE) of order n.
Ordinary differential equations are equations involving a function and some of its derivatives. Some approaches to find solutions require the equation in such a special form that
the highest derivative is isolated on one side of the equation.
Definition 2.2 (Implicit and explicit form)

An ordinary differential equation
in the general form
F (t, y, y , y , . . . , y (n) ) = 0
is called an ordinary differential equation in implicit form.
in the special form
y (n) = f (t, y, y , y , . . . , y (n1) )
is called an ordinary differential equation in explicit form.
Linearity is also a fundamental property of an ordinary differential equation. The set of
all solutions then can be calulated always with a fixed scheme.
Definition 2.3 (Linear ordinary differential equation)

An ordinary differential equation is called linear if y and all its derivaties appear only
as linear terms.
9
Example 2.1 (Ordinary differential equations)
a) The equation y = y is an explicit, first-order, linear equation. The general solution reads
y(t) = c et with c R.
b) The equation y = y is an explicit, second-order, linear equation with constant coefficients.

The set of all solutions is given by y(t) = c1 sin t + c2 cos t with c1 , c2 R.
c) The second-order, linear equation
2y + 4t2 y 8y = 0
y = 2t2 y + 4y
can be rewritten in explicit form.

d) The equation y 7y sin t = 0 is a third-order, linear equation in implicit form.
e) The second-order, non-linear equation y = y 2 + 3 sin t is in explicit form.
f) The first-order, non-linear equation y + 3 cos y = 8 y 2 cannot be transformed into an explicit

form.
The general solution of an ordinary differential equation typically is a set of infinite many
functions. Depending on appropriate additional initial conditions the solution then is
unique.
Definition 2.4 (Initial value problem)

An ordinary differential equation of order n for
a function y(t) together with n initial conditions (IC) at one single point t0 is called an
initial value problem (IVP):
n=2
y0
y(t0 )
= y0
y (t0 )
= y1
= yn1
y (n1) (t0 )
t0
Example 2.2 (Initial value problems)

a) Using the general solution from Example 2.1 the initial value problem has the following
unique solution:
y = y,
y(0) = 2
y(t) = 2 et .
b) Using the set of all solutions from Example 2.1 the initial value problem has the following
unique solution:
y = y,
y(0) = 1,
y (0) = 0
y(t) = cos t .
For boundary conditions the situation is different. The existence and uniqueness of solu-
10
tions is not closely related to the number of boundary values.
Definition 2.5 (Boundary value problem)

An ordinary differential equation of order n together with n boundary conditions (BC) at at
least two different points t0 and t1 is called a
boundary value problem (BVP):
y(t0 )
= y0
y(t1 )
= y1
y (t0 )
= y2
y (t1 )
= y3
y (m) (tk )
n=4
y1
y0
= yn1
t0
t1
Example 2.3 (Boundary value problems)

a) With the set of all solutions y(t) = c1 sin t+c2 cos t from Example 2.1 the following boundary
value problem has a unique solution:
y = y,
y(0) = 1,
y ( ) = 1
2
y(t) = sin t + cos t .
b) Using the general solution y(t) = c1 sin t + c2 cos t from Example 2.1 the following boundary
value problem has no solution:
y = y,
y(0) = 1,
y() = 0
c2 = 1,
c2 = 0 .
c) Again using the general solution y(t) = c1 sin t + c2 cos t from Example 2.1 the following
boundary value problem has infinite many solutions:
y = y,
y(0) = 1,
y() = 1
y(t) = c1 sin t + cos t .
Boundary value problems

Existence and uniqueness are more complicated for boundary value problems than for
initial value problems.
For first-order ordinary differential equation in explicit form the right-hand side of the
equation states the slopes of the solution curves. Using this information a vector field can
be drawn. Connecting the the small line elements gives the set of all solutions.
11
Definition 2.6 (Vector field, flow lines, line elements)

y
Let y = f (t, y) be a first-order ordinary differential equation. Then a set of small flow lines
or line elements at points (t, y) with slopes
f (t, y) is called a vector field of the differential equation.
t
Example 2.4 (Vector fields)
a) The vector field of the differential equation
y =
y
t
consists of line elements that are in the direction

of lines through the origin. No line elements are
defined on the y-axis. So, the solutions are half
lines that end at the origin.
b) The line elements of the differential equation

y =
t
y
are orthogonal to the line elements of y = yt . No

line elements are defined on the t-axis. Hence,
the solutions are half circles centered at the origin.
Theorem 2.1 (Vector field)

Each regular point in the vector field corresponds uniquely to one single solution.
Solutions cannot intersect.
Ordinary differential equations can be generalized from single equations to systems. Each
model state typically contributes one equation to the system. In general, the equations
are coupled. This means that more than one state is involved in each equation.
12
Definition 2.7 (System of ordinary differential equations)

A set of first-order ordinary differential equations
y
= f (t, y)
= f1 (t, y1 , . . . , yn )
= fn (t, y1 , . . . , yn )
y1
yn
is called a first-order system of ordinary differential equations.

If the right-hand side of a system of equations does not depend on the independent
variable t, the system is called autonomous. Then, the change in the states depends only
the states themselves and not on time. Autonomous systems are special systems.
Definition 2.8 (Autonomous system of ordinary differential equations)

A set of first-order ordinary differential equations is called autonomous if all functions
fk on the right-hand side do not depend on t:
y
= f (y)
y1
yn
= f1 (y1 , . . . , yn )
= fn (y1 , . . . , yn )
By introducing new states ordinary differential equation of higher order can be transformed
into first-order systems. This is important, because many numerical methods only deal
with first-order equations.
Transformation into a system
An ordinary differential equation of order n can be transformed into a system of n
first-order differential equations:
x(n) = f (x, x , . . . , x(n1) )
y1 = x
y1
y2 = x
y2
,
yn1 = x(n2)
yn1
(n1)
yn = x
yn
=
=
=
=
y2
y3
yn
f (y1 , . . . , yn )
13
Example 2.5 (Systems of differential equations)
a) The third-order differential equation
x 3tx + 5x = sin t
can be transformed into a system:
y1
y2
y3
=
=
=
x
x
x
y1
y2
y3
=
=
=
y2
y3
5y1 + 3ty2 + sin t
Since the differential equation is linear, the system also can be written using matrices:
y1 0
y2 = 0
y3 5
1
0
3t
0 y1 0
1 y2 + 0 .
0 y3 sin t
b) The nonlinear, second-order differential equation with initial values

x = cos(tx) + 4t2 + 6x,
x(0) = 2,
x (0) = 1
can be written as a nonlinear system:

y1
y2
=
=
x
x
y1
y2
=
=
y2
cos(ty1 ) + 4t2 + 6y1
,
,
y1 (0)
y2 (0)
=
=
2
1
=
=
=
=
0
1
4
2
c) The nonlinear second-order system with initial values

x
z
=
=
6xx z + tet
t + x + z + x + z
,
,
x(1)
z(1)
=
=
0
1
,
,
x (1)
z (1)
=
=
4
2
can be transformed into a first-order system

y1
y2
y3
y4
=
=
=
=
x
z
x
z
y1
y2
y3
y4
=
=
=
=
y3
y4
6y1 y3 y4 + tet
t + y1 + y2 + y3 + y4
,
,
,
,
y1 (1)
y2 (1)
y3 (1)
y4 (1)
14
Example 2.6 (Vibration equation)
The vibration equation is defined as
x
+ 2 x + 02 = 0 ,
where 0 is the damping constant and 0 > 0 is the angular frequency. Using the new states
y1 = x ,
y2 = x
the second-order differential equation can be transformed into a linear first-order system:
y 1
0
=
2
y 2 0
1 y1
2 y2
Transformation of a system into an autonomous system

A non-autonomous first-order system of ordinary differential equations can be transformed into an autonomous system using the time variable as an additional state
yn+1 = t:
y1
yn
= f1 (t, y1 , . . . , yn )
= fn (t, y1 , . . . , yn )
y1
yn
yn+1
= f1 (yn+1 , y1 , . . . , yn )
= fn (yn+1 , y1 , . . . , yn )
= 1
Example 2.7 (Autonomous system)

The third-order differential equation from Example 2.5
x 3tx + 5x = sin t
can be transformed into an autonomous system:
y1
y2
y3
y4
=
=
=
=
x
x
x
t
y1
y2
y3
y4
=
=
=
=
y2
y3
5y1 + 3y2 y4 + sin y4
1
Note, that the original differential equation is linear, whereas the autonomous system has lost
the property of linearity.
There are different types of solutions. A solution that does not change its state at all is
called an equilibrium solution. Other special solutions are periodic solutions. For periodic
solutions, the system states are repeated again and again.
15
Definition 2.9 (Equilibrium solution, periodic solution)

A solution of a system of ordinary differential equations in the form
y(t) = y 0 with y 0 Rn is called an equilibrium solution.
y(t) = y(t + T ) with T > 0 is called a periodic solution. The parameter T denotes
the period.
Example 2.8 (Types of solutions)

a) Consider again the differential equation y = y from Example 2.1. The function y(t) = 0 is
an equilibrium solution. All solutions y(t) = c1 sin t + c2 cos t are periodic.
b) Consider the differential equation y = y. The trivial function y(t) = 0 is an equilibrium

solution. No solution y(t) = c1 et + c2 et is periodic.
Definition 2.10 (Phase-plane, phase-space, phase portrait)

The plane or space can be used to visualize the dynamics of differential equations.
The plane in which autonomous, two-dimensional first-order systems can be analyzed is called phase-plane.
For higher dimensional systems the dynamics can be analyzed in the phase-space.
The phase-plane or phase-space together with some solution curves is called phase
portrait.
16
Example 2.9 (Phase plane)
y2
a) The initial value problem

x +
4
x = 0,
9
x (0) = 0 .
x(0) = 3,
can be transformed into a system:

y1
y2
y1 (0) = 3
y2
4
y1
9
3 2 1
1
y2 (0) = 0
y1
y1
The solution curve is an ellipse that is oriented counterclockwise:

2
y1 (t) = 3 cos ( t) ,
3
2
y2 (t) = 2 sin ( t) .
3
y2
b) Rewriting the initial value problem

x +
52 4
x + x = 0, x(0) = 3, x (0) = 0 .
255
9
yields the linear system

y1
y2
y1 (0) = 3
y2
52
4
y2
y1
9
255
y2 (0) = 0
3 2 1
1
2
The solution curve is a spiral that tends to the origin:

y1 (t) = 3 e 255 t cos (
26
168
t) ,
255
y2 (t) = 3 e 255 t (
26
168
168
168
26
cos (
t)
sin (
t)) .
255
255
255
255
Finally, we introduce the definition of stability. Stability of a solution in general means that
nearby solutions always converge towards the stable solution. This principle of stability is
especially important for equilibrium solutions.
Definition 2.11 (Stability of an equilibrium solution)

An equilibrium solution y(t) = y 0 of an autonomous system is called stable if any
solution in the neighborhood of y 0 will always approach this equilibrium solution as
t . In this case y(t) y 0 for t . Otherwise the equilibrium solution is called
unstable.
17
2.1.2 Numerical Basics

This section is about the very basics of numerical solutions for ordinary differential equations. Many numerical methods generate sequences of points as solutions. The numerical
solution is calculated only at discrete points.
Definition 2.12 (Discretization)

The transformation of a continuous solution
y(t) for t [0, ) into a discrete sequence of
points (yk ) for k N0 at points in time tk with
y
wk
y(t)
yk = y(tk ) wk
is called discretization. If the points in time
tk are equidistant with h > 0 then tk = hk.
Numerical approximations of y are denoted by
w.
yk = y(tk )
t
The error of numerical solutions has two main sources. Firstly, any numerical algorithm
is an approximation scheme. For instance, derivatives can be substituted by difference
formulas. Such discretizations cause truncation errors. Secondly, on computers, numbers
are stored with a fixed number of digits. Any floating-point operation thus causes roundoff
d
errors. As a rule of thumb, there optimal step size is h0 = 10 2 , where d is the number
of digits. On general purpose personal computers, d = 16 and thus h0 = 108 . For many
real-world applications, step sizes between 105 and h = 103 yield solutions that are
sufficiently accurate.
Definition 2.13 (Truncation error, roundoff error)

The error of a numerical method caused by

discretization is called truncation error,

the limited machine precision is called

roundoff error.

Both errors depend on the step size h and contribute to the total error.

The approximation quality of numerical solutions is a central issue in the context of initial
value problems. The most important error is the global error that describes the difference
between the exact and the numerical solution. In most cases, this global error is very
hard to calculate or estimate. As a substitute, the local error is defined as the difference
between numerical and exact solution in one single step. This local error is much easier
18
to compute.
Definition 2.14 (Local and global truncation error)

Given a numerical integration method.
The local truncation error E(t, h) at time t with step size h is defined as the
difference between the numerical method applied to the exact solution and the
exact solution itself.
The global truncation error EG (t, h) at time t with step size h is defined as the
difference between the numerical solution w(t, h) and the exact solution y(t):
EG (t, h) = w(t, h) y(t) .
A numerical integration method should be at least consistent. This minimum requirement

ensures that the numerical approximation solves the given problem and not any other
one. The order of a numerical integration method describes the speed of convergence. In
general, the higher the order, the more accurate is the solution.
Definition 2.15 (Consistence, convergence, order)

A numerical integration method with step size h, global truncation error EG (t, h) and
local truncation error E(t, h) is called
consistent if the local truncation error devided by the step size tends to zero:
1
E(t, h) 0
h
(h 0)
t ,
of order p > 0 if the local truncation error is bounded by h(p+1) :

E(t, h) = O(hp+1 )
t ,
convergent if the global truncation error tends to zero:

EG (t, h) 0 (h 0)
t .
19
2.2 Eulers Method and Variants

In this section, we discuss simple numerical integration schemes. They are easy to understand, quick to program, and helpful to get a first impression of the solution. Of course,
quite small step sizes are necessary to achieve a reasonable precision.
2.2.1 Eulers Method

The simplest approach to solve a first-order ordinary differential equation numerically is
to substitute the derivative by the forward difference formula:
y (t)
y(t + h) y(t)
.
h
Let wk denote the numerical solution at tk = kh with step size h. Then, inserting the
forward difference formula into the differential equation yields
wk+1 wk
= f (tk , wk ) .
h
Resolving this equation with respect to the next iteration wk+1 is called Eulers method.
Definition 2.16 (Eulers method)

Let y = f (t, y) be a first-order differential
equation. Then, Eulers method generates a
solution (wk ) at points in time tk = t0 +hk with
step size h beginning with w0 = y0 and using
wk+1

= wk + h f (tk , wk )

for k N0 . The increment function is the

slope at the left point of each interval.

To investigate the order of Eulers approach, we discuss the local truncation error. We
insert the exact solution y into Eulers formula and get
E(t, h) = y(t) + hf (t, y(t)) y(t + h) .

exact
Euler
Now we use Taylor series expansion and obtain
E(t, h) = (y(t) + hf (t, y(t))) (y(t) + hy (t) + O(h2 )) = O(h2 ) .
Since y (t) = f (t, y) the leading expressions disappear.
20
Theorem 2.2 (Eulers method)

Eulers method is consistent and of order 1. Each iteration step requires 1 function
evaluation.
2.2.2 Implicit Eulers Method

The idea of the implicit variant of Eulers method is to use the backward difference
formula:
y(t + h) y(t)
y (t + h)
.
h
Let wk denote the numerical solution at tk = kh with step size h. Then, inserting the
backward difference formula into the differential equation yields
wk+1 wk
= f (tk+1 , wk+1 ) .
h
This formula is called Eulers method.
Definition 2.17 (Implicit Eulers method)

Let y = f (t, y) be a first-order differential equation. Then, the implicit Eulers
method generates a solution (wk ) at points in time tk = t0 + hk with step size h
beginning with w0 = y0 and using
wk+1
= wk + h f (tk+1 , wk+1 )

for k N0 . The increment function is the slope at the right point of each interval.
We discuss the order of the implicit Euler: Inserting the exact solution y into the formula
for the local truncation error yields
E(t, h) = y(t) + hf (t + h, y(t + h)) y(t + h) .

exact
implicit Euler
Now, we use Taylor series expansion and obtain
E(t, h) = (y(t) + h (f (t, y(t)) + O(h)) ) (y(t) + hy (t) + O(h2 )) = O(h2 ) .
Since y (t) = f (t, y) the leading expressions disappear.
Theorem 2.3 (Implicit Eulers method)
The implicit Eulers method is consistent and of order 1. Each iteration step requires
the solution of a nonlinear equation. In contrast to explicit methods this method has
a large region of stabiliy.
21
2.2.3 Improved Eulers Method

Eulers method uses the slope at the end points of the interval [t, t + h]. To improve
accuracy, the slope at the midpoint of this interval can be used as well. This gives an
improved version of Euler.
Definition 2.18 (Improved Eulers method)

Let y = f (t, y) be a first-order differential equation. Then, the improved Eulers
method generates a solution (wk ) at points
in time tk = t0 + hk with step size h beginning
with w0 = y0 and using
K1
= f (tk , wk )
K2
= f (tk + 21 h, wk + 12 h K1 )
wk+1

= wk + h K2
for k N0 . The increment function is the slope at the midpoint of each interval.
To discuss the order of the improved Euler, we insert the exact solution y into the formula
of the local truncation error:
1
1
h, y(t + h) + hf (t, y(t))) y(t + h) .
2
2

exact
improved Euler
E(t, h) = y(t) + hf (t +
For simplicity, we now omit the argument (t) for function y and the arguments (t, y(t))
for function f and its derivatives. Again, using Taylor series expansion yields
E(t, h) =
1
h (ft + fy (f + O(h))) + O(h2 )))
2
1
(y + hy + h2 y + O(h3 )) = O(h3 ) .
2
(y + h (f +
The derivative of y (t) = f (t, y(t)) with respect to t is obtained by the general chain rule.
Since y = f and y = ft + fy f , the leading expressions disappear.
Theorem 2.4 (Improved Eulers method)
The improved Eulers method is consistent and of order 2. Each iteration step requires
2 function evaluations.
22
2.2.4 Heuns Method

The idea of Heuns approach is to average both slopes at the end points of the interval [t, t + h]. This also gives an improved version of Euler. The precision achieved is
comparable to that of the improved Eulers method.
Definition 2.19 (Heuns method)

Let y = f (t, y) be a first-order differential
equation. Then, Heuns method generates a
solution (wk ) at points in time tk = t0 +hk with

K1
= f (tk , wk )
K2
= f (tk + h, wk + h K1 )
wk+1
= wk + h
K1 + K2 )

( 21

1
2
for k N0 . The increment function is the average of the slopes at the left and right
point of each interval.
The formula of the local truncation error has to used to proof the order of Heuns method:
1
1
E(t, h) = y(t) + h ( f (t, y(t)) + f (t + h, y(t) + hf (t, y(t)))) y(t + h) .
2
2

exact
Heun
Again, for simplicity, we now omit the arguments (t) and (t, y(t)). Using Taylor series
expansion yields
E(t, h) =
1
1
hf + h (f + h (ft + fy (f + O(h))) + O(h2 )))
2
2
1 2
(y + hy + h y + O(h3 )) = O(h3 ) .
2
(y +
The derivative of y (t) = f (t, y(t)) with respect to t is obtained by the general chain rule.
Since y = f and y = ft + fy f , the leading expressions disappear.
Theorem 2.5 (Heuns method)
Heuns method is consistent and of order 2. Each iteration step requires 2 function
evaluations.
23
2.3 Runge-Kutta Methods

Runge-Kutta methods represent a very useful and popular class of one-step methods.
2.3.1 Runge-Kutta Methods

Definition 2.20 (Classical Runge-Kutta method)
Let y = f (t, y) be a first-order differential equation. Then, the classical RungeKutta method generates a solution (wk ) at points in time tk = t0 + hk with step size
h beginning with w0 = y0 and using
K1 = f (tk , wk )

K2 = f (tk + 12 h, wk + 21 h K1 )
K3 = f (tk + 21 h, wk + 21 h K2 )
K4 = f (tk + h, wk + h K3 )
wk+1 =
wk +h ( 16 K1 + 62 K2 + 26 K3 + 61 K4 )

for k N0 . The increment function is an average of two slopes at the left and right
point and two slopes at the midpoint of each interval.
Theorem 2.6 (Classical Runge-Kutta method)
The classical Runge-Kutta method is consistent and of order 4. Each iteration step
requires 4 function evaluations.
24
Definition 2.21 (General Runge-Kutta method)

Let y = f (t, y) be a first-order differential equation. Then, the r-stage general RungeKutta method generates a solution (wk ) at points in time tk = t0 + hk with step size
h beginning with w0 = y0 and using
i1
= f (t + ai h, y + h bi,j Kj )
Ki
j=1
= wk + h ci Ki
wk+1
i=1
for k N0 . The coefficients can be listed in a table:

a1
a2
a3
ar
b2,1
b3,1
br,1
c1
b3,2
br,2
c2
...
...
br,r1
cr1
cr
Theorem 2.7 (General Runge-Kutta method)

The maximum order p of a general Runge-Kutta method depends on the stages r:
stage r
order p
1
1
2
2
3
3
4
4
5
4
6
5
7
6
8
6
9
7
Special Runge-Kutta methods of order 2

The coefficient tables of special Runge-Kutta methods of order 2 are:
0
0
1
2
1
2
1
1
Improved Eulers method
1
1
2
1
2
Heuns method
25
0
1
2
1
2
1
6
4
6
1
6
Kuttas method

0
1
2
1
2
1
2
1
2
1
6
2
6
2
6
1
6
Classical RK method
1
3
2
3
1
3
13
1
8
3
8
3
8
3/8 method
1
8
26
2.3.2 Step Size Control

Definition 2.22 (Step size control)
The principle of step size control can be applied using two numerical methods of
different order or of different discretization. The global error can be estimated by the
difference of two numerical solutions (wk ) and (w
k ):
E = wk+1 w
k+1 .
The step size h is adapted in dependence of a given error bound :
If E > h then redo the computation of wk+1 with
If E <
1
4
1
2
h.
h then accept wk+1 and set the step size to 2h for the next step.
Otherwise accept wk+1 and continue.

The numerical solution (w
k+1 ) is used only to control the step size.
Special embedded Runge-Kutta methods of order 2/3
The coefficient tables of special embedded Runge-Kutta methods of order 2/3 are:
0
0
1
2
1
2
1
2
1
4
1
2
1
6
1
6
4
6
1
6
Embedded improved Eulers method
1
4
1
2
1
6
4
6
Embedded Heuns method
27
Definition 2.23 (Runge-Kutta-Fehlberg method)

Let y = f (t, y) be a first-order differential equation. Then, the embedded 6-stage
Runge-Kutta-Fehlberg method of order 4/5 with step size control generates a solution (wk ) using the coefficients
0
1
4
3
8
12
13
1
1
2
1
4
3
32
1932
2197
439
216
8
27
25
216
16
135
9
32
7200
2197
8
2
0
0
7296
2197
3680
513
3544
2565
1408
2565
6656
12825
845
4104
1859
4104
2197
4104
28561
56430
11
40
51
9
50
2
55
2.3.3 The One-Step Approach

Definition 2.24 (One-step method, increment function)
A numerical integration method that generates a solution (wk ) at points in time (tk )
with step size h of the form
a0 wk + a1 wk+1 = h (tk , tk+1 , wk , wk+1 )
with coefficients ai is called one-step method. The function is called increment
function.
28
2.4 Multistep Methods

Runge-Kutta methods all have in common that they use only the last iterate to calculate
the next one. As an alternative, more than one old value can integrated into a numerical
algorithm. This idea leads to multistep methods. They use several old iterates to compute
the next one.
2.4.1 Linear Multistep Methods

The special approach of linear multistep methods uses a linear combination of the function
values f (tk+i , wk+i ) to build up the incremental function. It is important to mention the
fact, that linear multistep methods are not restricted to linear differential equations. The
linearity refers to the function values in the formula.
Definition 2.25 (Linear multistep methods)

Let y = f (t, y) be a first-order differential equation. Then, a linear multistep method
or linear m-step method generates a solution (wk ) at points in time tk = t0 + hk with
m
ai wk+i
i=0
= h bi f (tk+i , wk+i )
i=0
for k N0 . The coefficients with a20 + b20 0 and a2m + b2m 0 can be listed in a table:
a0
b0
a1
b1
...
...
am
bm
A method is explicit if bm = 0 and implicit otherwise. The first iterates w1 , . . . , wm1

are computed with a one-step method.
Runge-Kutta methods can be compared with multistep methods. Characteristical features
are the number of function calls, the achieved order, or the possibility to adapt the step
size. Since multistep methods need several historic values, they always are accomplished
with one-step methods with which the first iterates are computed.
29
Linear multistep methods
Linear multistep methods have some characteristics in common:
The general advantages are high order and high accuracy with few computational
effort.
The general disadvantage are the limitation to constant step size and the necessity
for seed computation for the first iterates.
2.4.2 Adams-Bashforth Methods

A very popular class of linear multistep variants are the so-called Adams-Bashforth methods. They are named after the English mathematicians John Couch Adams and Francis
Bashforth. They all have in common that only the last two coefficients am1 = am = 1
are nonzero. Moreover, this class contains only explicit methods with bm = 0.
Definition 2.26 (Adams-Bashforth methods)
Explicit linear multistep methods with
a0 = a1 = . . . = am2 = 0
am1 = 1
am = 1
bm = 0
are called Adams-Bashforth methods (AB).

Investigating the local truncation error a set of possible method variants can be derived.
The achievable order depends on the number of included old values. Eulers method can
be regarded as a special Adams-Bashforth method. The general approach for m is
wk+2 = wk+1 + h(b0 f (tk , wk ) + b1 f (tk+1 , wk+1 )) .
Inserting this approach into the local truncation error formula yields
E = y(t + h) + h(b0 f (t, y(t)) + b1 f (t + h, y(t + h))) y(t + 2h) .
A Taylor expansion with center point t + h leads to the error representation
E = y + h(b0 (f h(ft + fy f ) + O(h2 )) + b1 f ) (y + hy +
1 2
h y + O(h3 )) .
2
Here, the arguments (t) and (t, y(t)) are omitted. Comparing all expressions with y = f
and y = ft + fy f , respectively, the following equations result:
hb0 + hb1 h = 0
1
h2 b0 h2 = 0 .
2
30
Dividing by h and h2 and rearranging the expressions yields
b0 + b1 = 1
b0 =
1
.
2
This system of linear equations has the solution b0 = 21 und b1 = 32 .

Adams-Bashforth methods
The coefficients of m-step Adams-Bashforth methods together with the order p are:
m
am1
am
12
5
12
9
24
251
720
3
2
16
12
37
24
1274
720
b0
b1
b2
b3
b4
b5
p
1
0
23
12
59
24
2616
720
2
0
55
24
2774
720
3
4
0
1901
720
The special case p = 1 is Eulers method.

Example 2.10 (Adams-Bashforth Methods)
a) For m = 1, we obtain Eulers method as a special case of an Adams-Bashforth variant:
wk+1 = wk + hf (tk , wk ) .
b) For m = 2, a possible Adams-Bashforth variant with p = 2 is
3
1
wk+2 = wk+1 + h ( f (tk , wk ) + f (tk+1 , wk+1 )) .
2
2
2.4.3 Adams-Moulton Methods

The implicit analogon to Adams-bashforth methods are Adams-Moulton methods. This
class of multistep variants are named after the English and American mathematicians
John Couch Adams and Forest Ray Moulton. Again, only the last two coefficients
am1 = am = 1 are nonzero. In contrast, now bm 0, which implies implicit formulas.
Definition 2.27 (Adams-Moulton methods)
Implicit linear multistep methods with
a0 = a1 = . . . = am2 = 0
am1 = 1
are called Adams-Moulton methods (AM).
am = 1
bm 0
31
To get appropriate coefficients bk , the local truncation error is stated with the objective to
cancel out as many leading expressions as possible. As in the explicit case, the achievable
order depends on the number of included old values. The implicit Eulers method is a
special Adams-Moulton method.
Adams-Moulton methods
The coefficients of m-step Adams-Moulton methods together with the order p are:
m
am1
am
1
2
1
12
1
24
19
720
1
2
8
12
5
24
106
720
1
1
1
b0
b1
b2
b3
b4
p
1
2
5
12
19
24
264
720
3
9
24
646
720
4
251
720
The special case p = 1 is the implicit Eulers method. The special case p = 2 is the
trapezoidal method.
Example 2.11 (Adams-Moulton Methods)
a) For m = 1, we obtain the implicit Eulers method of order 1:
wk+1 = wk + hf (tk , wk+1 ) .
b) Again for m = 1, there is a another Adams-Moulton method. This method is called trapezoidal method and has order 2:
1
1
wk+1 = wk + h ( f (tk , wk ) + f (tk+1 , wk+1 )) .
2
2
2.4.4 Predictor-Corrector Methods

Definition 2.28 (Predictor corrector methods)
Multistep predictor corrector methods (PC) or Adams-Bashforth-Moulton methods (ABM) are defined as follows:
(1) Compute predictor wk+m

with explicit Adams-Bashforth m-step method.
(2) Compute corrector wk+m with implicit Adams-Moulton m 1-step method using
wk+m
in the right-hand side of the Adams-Moulton scheme.
32
Predictor corrector methods
The advantage of predictor corrector methods is that there is no need for rootfinding
subroutines. Using the predictor in the right-hand side of the scheme transforms a
nonlinear problem into a linear one.
2.4.5 The General Multistep Approach

The linear multistep approach can be generalized into a nonlinear variant. Then, the
right-hand side of the multistep method formula has nonlinear incremental function of
any possible form.
Definition 2.29 (Multistep method, increment function)

A numerical integration method that generates a solution (wk ) at points in time (tk )
with step size h of the form
m
ai wk+i = h (tk , . . . , tk+m , wk , . . . , wk+m )
i=0
with coefficients ai is called multistep method or m-step method. The function

is called increment function.
33
2.5 Stability and Stiffness

There are several aspects that contribute to a good numerical solution. Of course, precision and accuracy are among these aspects. Moreover, a weaker requirement is that
a numerical solution shows the same tendency as the exact solution. Stiff differential
equations are a special challenge for numerical integration methods. They require small
iteration steps although the solution is quite smooth.
2.5.1 Stability of One-Step Methods

If the exact solution of a differential equation tends to zero, the same behavior is expected
from the numerical solution. This requirement ensures that the numerical method is
consistent with the differential equation and thus behaves stable. In general, stability is
achieved only for step sizes below critical bounds. To investigate the issue of stability, we
consider the simple linear test differential equation
y = y
C.
Since most real-world models embody friction or damping, the interesting case is Re() < 0.
Applying a one-step method with step size h to this equation generates a recursive formula
of the form
wk+1 = g(h)wk .
for the numerical solution wk . Here, g(z) denotes some complex function with a complex
argument z = h C. If Re() < 0, the exact solution y(t) tends to zero. The numerical
solution wk has the same behavior, if g(z) < 1.
Definition 2.30 (Region of stability for one-step methods)
Consider the test differential equation y = y. For a one-step method with a numerical
solution (wk ) and a step size h of the form
wk+1 = g(h)wk
the region of stability is defined as that part of the complex plane where g is contracting:
G = {z C g(z) < 1} .
Within G and the left-half plane the numerical solution has the same quantitative
behavior as the exact solution.
34
Boundary of region of stability for one-step methods
The boundary of the region of stability consists of all solutions z of the equations
g(z) = ei ,
0 < 2 ,
where g results from the one-step method applied to the test differential equation.
Example 2.12 (Stability of Eulers method)
Applying Eulers method
wk+1 = wk + h f (tk , wk )
to the test differential equation y = y yields
wk+1 = wk + hwk = (1 + h)wk .
Using z = h, the stability function reads
g(z) = 1 + z .
The boundary of the region of stability is defined as g(z) = 1. Since all complex points on the
unit circle are of the form ei , we get
1 + z = ei
z = 1 + ei .
With [0, 2] the boundary of the region of stability describes a circle with center 1 and
radius 1. The region of stability lies within the circle.
35
Example 2.13 (Stability of improved Eulers and Heuns method)
a) Applying the improved Eulers method
wk+1 = wk + h f (tk +
1
1
h, wk + h f (tk , wk ))
2
2

wk+1 = wk + h (wk +
1
1
1
h f (tk , wk )) = wk + h (wk + hwk ) = (1 + h + h2 2 ) wk .
2
2
2

g(z) = 1 + z +
1 2
z .
2
The boundary of the region of stability is defined as g(z) = 1. Since all complex points on
the unit circle are of the form ei , we get
1+z+
1 2
z = ei
2
z1,2 = 1
2 ei 1 .
With [0, 2] the boundary of the region of stability describes a closed curve which looks
like an oval. For special values we obtain
= 0 z = 2, 0 , = z = 1 3i .
b) Applying Eulers method
1
1
wk+1 = wk + h ( f (tk , wk ) + f (tk + h, wk + hf (tk , wk)))
2
2
1
1
1
wk+1 = wk + h ( wk + (wk + hwk )) = (1 + h + h2 2 ) wk .
2
2
2
Hence, the stability function of Heuns method
g(z) = 1 + z +
1 2
z .
2
is the same as for the improved Eulers method.
36
Example 2.14 (Stability of Kuttas method)
Applying Kuttas method to the test differential equation y = y yields
K1
f (tk , wk )
wk
K2
f (tk +
(1 +
K3
f (tk + h, wk + h(K1 + 2K2 ))
(1 + h + h2 2 ) wk .
1
2
h, wk + h 12 K1 )
1
2
h) wk
All together we obtain

wk+1
wk + h( 61 wk + 46 (1 +
(1 + h +
1
2
h2 2 +
1
6
1
2
h) wk + 16 (1 + h + h2 2 ) wk )
h3 3 ) wk .

g(z) = 1 + z +
1 2 1 3
z + z .
2
6
1+z+
1 2 1 3
z + z = ei .
2
6
This equation can be solved either numerically or with a formula for the roots of cubic polynomials.
With [0, 2] the boundary of the region of stability describes a closed curve in the complex
plane.
Example 2.15 (Stability of classical Runge-Kuttas method)

Applying the classical Runge-Kuttas method to the test differential equation y = y yields the
stability function
g(z) = 1 + z +
1 2 1 3 1 4
z + z +
z .
2
6
24
1+z+
1 2 1 3 1 4
z + z +
z = ei .
2
6
24
This equation can be solved either numerically of with a formula for the roots of cubic polynomials.
With [0, 2] the boundary of the region of stability describes a closed curve in the complex
plane.
37
Stability of Runge-Kutta methods
Runge-Kutta methods are explicit methods.
The regions of stability are bounded and cover
only small parts of the left-half plane.
Im
RK1: Euler
6 5 4 3 2 1
i
RK2: Improved Euler and Heun
RK4
RK3
RK2
RK1
1
Re
RK3: Kutta
RK4: Classical Runge-Kutta
In terms of stability, good numerical methods cover a wide range of the left-half plane.
There are a couple of methods that cover the complete left-half plane. Such methods are
called A-stable. They are optimal with respect to stability.
Definition 2.31 (A-stability)

An integration method is called A-stable if the region of stability G covers the complete
left-half plane:
G {z C Re(z) < 0} .
Example 2.16 (Stability of implicit Eulers method)

Applying Eulers method
wk+1 = wk + h f (tk+1 , wk+1 )
wk+1 = wk + hwk+1
wk+1 =
1
wk .
1 h

g(z) =
1
.
1z
1
= ei
1z
z = 1 ei .
With [0, 2] the boundary of the region of stability describes a circle with center 1 and radius
1. The region of stability lies outside the circle and thus covers the whole left half plane. The
implicit Eulers method is A-stable. This is a big advantage of this approach.
38
Example 2.17 (Stability of trapezoidal method with one-step definition)
Applying the trapezoidal method
1
1
wk+1 = wk + h ( f (tk , wk ) + f (tk+1 , wk+1 ))
2
2
1
1
wk+1 = wk + h ( wk + wk+1 )
2
2
wk+1 =
2 + h
wk .
2 h

g(z) =
2+z
.
2z
2+z
= ei
2z
z=2
ei 1
2 sin
=
i.
ei + 1 1 + cos
The last equation results from expanding the fraction with the complex conjugate of the denominator ei + 1 and using the relations
sin =
1 i
(e ei )
2i
cos =
1 i
(e + ei ) .
2
With [0, 2] the boundary of the region of stability is the imaginary axis. The region of
stability lies left to the imaginary axis and thus covers the whole left half plane. The trapezoidal
method is A-stable.
Definition 2.32 (Characteristic polynomials)

The coefficients of a multistep method can be used to build up two polynomials:
m
() = ai i ,
i=0
() = bi i .
i=0
are called the first characteristic polynomial and the second characteristic polynomial of the given multistep method.
39
Example 2.18 (Characteristic polynomials of Adams methods)
a) The characteristic polynomials of the Adams-Bashforth method with m = 1 are
() = 1 +
() = 1 .
b) The characteristic polynomials of the Adams-Bashforth method with m = 2 are

() = + 2
1 3
() = + .
2 2
c) The characteristic polynomials of the Adams-Bashforth method with m = 3 are

() = 2 + 3
() =
16
23
5
+ 2 .
12 12
12
d) The characteristic polynomials of the Adams-Moulton method with m = 1 and p = 1 are

() = 1 +
() = .
e) The characteristic polynomials of the Adams-Moulton method with m = 1 and p = 2 are

() = 1 +
() =
1 1
+ .
2 2
f) The characteristic polynomials of the Adams-Moulton method with m = 2 are

() = + 2
() =
8
5
1
+ 2 .
12 12
12
2.5.2 Stability of Multistep Methods

Once again, we consider the test differential equation
y = y
with Re() < 0. With this equation, a linear multistep method with step size h reads
m
i=0
i=0
ai wk+i = h bi wk+i
(ai zbi )wk+i = 0 ,
i=0
where z = h. This is a linear difference equation for the sequence wk . The general
solution of that difference equation is based on the powers
k , where
are the zeros of
the polynomial
m
p() = (ai zbi )i = 0 .

i=0
Hence, the general solution tends to zero, if all zeros

lie within the unit circle.
40
Definition 2.33 (Region of stability for linear multistep methods)

Consider the test differential equation y = y. For a multistep methods with a numerical solution (wk ) and a step size h of the form
m
i=0
i=0
ai wk+i = h bi wk+i
the region of stability is defined as that part of the complex plane where the stability
polynomial p has only zeros within the unit circle:
m
G = {z C p() = (ai zbi )i has only zeros

with
< 1 } .
i=0
Within G and the left-half plane the numerical solution has the same quantitative
behavior as the exact solution.
The boundary of the region of stability is reached if one zero
crosses the unit circle:
= 1
= ei
[0, 2) .
Inserting this value into the characteristic polynomial yields the boundary points
m
i = 0
(ai zbi )
i=0
z() =
(ei )
(ei )
Hence, all boundary points z() are obtained by varying in the interval [0, 2).
Boundary of region of stability for linear multistep methods
Using the characteristic polynomials and of a multistep method, the boundary of
the region of stability consists of all points
z=
(ei )
(ei )
0 < 2 .
Example 2.19 (Stability of Eulers method with multistep definition)

Using the characteristic polynomials, we obtain the boundary points of the region of stability:
z=
1 +
= 1 + ei .
1
This expression is the same as in Example 2.12. Again, all points z lie on a circle with center 1
and radius 1.
41
Stability of Adams-Bashforth methods
Adams-Bashforth methods are explicit methods. AB1 is Eulers method. The regions of
stability are bounded and cover very small parts
of the left-half plane. With increasing order the
regions are getting even smaller. A point that
belongs to any stability region is plotted in light
red.
Im
AB1
AB2
i
6 5 4 3 2 1
i
AB4
1
Re
AB3
Example 2.20 (Stability of trapezoidal method with multistep definition)

Using the characteristic polynomials, we obtain the boundary points of the region of stability:
z=
1 +
ei 1
.
= 2 i
1
e +1
+2
2
This expression is the same as in Example 2.17. Again, all points z lie on the imaginary axis.
Thus, the trapezoidal method is A-stable.
Stability of Adams-Moulton methods

Adams-Moulton methods are implicit methods.
AM1 is the implicit Eulers method. The region of stability of AM1 is the area outside the
unit circle with center z = 1. The region of
stability of AM2 is the left half of the plane.
The regions of stability of AM3 and AM4 are
bounded. AM1 and AM2 are A-stable. A point
that belongs to any stability region is plotted in
light red.
Im
AM2
AM1
AM3
AM4
6 5 4 3 2 1
i
Re
2.5.3 Stiffness
Stiffness describes the relation between necessary step size and solution smoothness. If
a solution behaves continuous and smooth without sudden changes of the slopes on one
hand and requests quite a small step size on the other hand, such a paradox situation is
called stiff. Stiffness is a matter of performance, not of precision. Stiffness also is a local
matter. A differential equation can behave normal in some region and stiff in another.
42
Definition 2.34 (Stiffness)

An ordinary differential equation y = f (t, y) is called stiff at (t, y)
if the solution being sought varies slowly, but there are nearby solutions that vary
rapidly, so the numerical method must take small steps to obtain satisfactory precision or
if there is an Eigenvalue of f y (t, y) with Re() << 0 or
if the Eigenvalues of f y (t, y) have very different negative parts.
Example 2.21 (Vibration equation with stiff behavior)
Using the special values 0 = 1 and = 5.05 in the
vibration system of Example 2.6 the system reads
y = Ay
with
0
A=(
1
1
) .
10.1
y
1
z1 (t)
0.5
The characteristic polynomial of A
z2 (t)
det(A I) = 2 + 10.1 + 1
1
has two zeros which generate the basis functions:

1 = 0.1 ,
2 = 10
z1 (t) = e0.1 t ,
z2 (t) = e10 t .
The corresponding eigenvector equations are

(A 1 I)v1 = (
0.1
1
1
) v1 = 0,
10
(A 2 I)v2 = (
10
1
1
) v2 = 0
0.1
with eigenvectors
v1 = (
10
) ,
1
v2 = (
1
) .
10
Using constants C1 and C2 , we obtain the general solution

y(t) = C1 z1 (t) v 1 + C2 z2 (t) v 2 .
Since 1 and 2 are very different, the system behaves stiff. In comparison with z1 , the function
z2 contributes almost nothing to the general solution.
43
Example 2.22 (Vibration equation solved with Euler)
We consider again the vibration equation of Example 2.6 with the parameters of Example 2.21. For the
initial value
1
) ,
y(0) = (
0
y
1
y1 (t)
= 0.17
h
= 0.23
h
0.5
the constants in the general solution are

C1 =
10
,
99
C2 =
10
.
99
Using Eulers method, the limit stability number is z = h = 2. This implies

h1 =
2
2
=
= 20 ,
1 0.1
h2 =
2
2
=
= 0.2 ,
2 10
h = min{h1 , h2 } = h2 = 0.2 .
The absolute value of eigenvalue 1 is much smaller than that of eigenvalue 2 . So, the limit
step size h1 derived from 1 is much larger than the limit step size h2 derived from 2 . Although
= 0.17 < h the
the solution part z1 is dominant, the solution part z2 limits the step size. With h
solution behavior is generated correctly. In contrast, with h = 0.23 > h, strong oscillations occur
in the numerical solution. This effect is called stiffness.
44
2.6 Initial Value Problems

Initial value problems can be solved numerically in many different ways. Most popular
methods integrate from the initial point in time to the end point step by step. The
resulting solution is a sequence of approximation points.
2.6.1 Summary
Initial value problems can be solved numerically in many different ways. Most popular
methods integrate from the initial point in time to the end point step by step. The
resulting solution is a sequence of approximation points.
2.6.2 Matlab Functions

Matlab provides a variety of built-in functions to solve ordinary differential euqations with
given initial values. These functions differ in precision, efficiency, and applicability to stiff
problems. One-step, multistep, explicit, and implicit methods all have their advantages
and disadvantages.
MATLAB solvers
MATLAB has a variety of built-in solvers for initial value problems:
solver
ode23
ode45
ode113
ode15s
ode15i
ode23s
ode23t
ode23tb
problem type
nonstiff
nonstiff
nonstiff
stiff
fully implicit
stiff
mildly stiff
stiff
type of algorithm
explicit Runge-Kutta pair, orders 2 and 3
explicit Runge-Kutta pair, orders 4 and 5 (workhorse!)
explicit linear multistep, orders 1 to 13
implicit linear multistep, orders 1 to 5
implicit linear multistep, orders 1 to 5
modified Rosenbrock pair (one-step), orders 2 and 3
trapeziodal rule (implicit), orders 2 and 3
implicit Runge-Kutta-type algorithm, orders 2 and 3
45
2.7 Boundary Value Problems

Ordinary differential equations appear together with initial or boundary values. If information about the solution is known at at least two points, the problem is called boundary
value problems. Calculating the bend of a fixed rod with load is a classical boundary value
problem. Another famous application is the re-entry problem of a space shuttle from the
orbit into the atmosphere.
2.7.1 Shooting Methods

Numerical methods for initial value problems can be used for boundary value problems as
well. Typically, the slope at the initial point is varied. This slope is adjusted such that
the difference between the numerical solution at the end point and the given boundary
value becomes as small as possible.
Definition 2.35 (Shooting method)

Boundary value problems can be solved with shooting methods:
(1) Transform the boundary value problem
y = f (t, y, y )
y(a) =
y(b) =
y (a) = s .
into an initial value problem:

y = f (t, y, y )
y(a) =
(2) Define the shooting function

g(s) = y(b, s)
(3) Find a root of the function g. Use Newtons method or the secant formula.
2.7.2 Finite Difference Methods

The idea of the finite difference approach to boundary value problems is to discretize
the integration interval and substitute the derivatives by difference formulas. Typically,
this yields systems of linear equations. The solution at all discrete points is calculated
simultaneously. this is in contrast to the shooting approach, where the solution points are
calculated from the starting point to the end point in an iterative process.
46
Definition 2.36 (Finite difference method)

Boundary value problems can be solved with finite difference methods:
(1) State the boundary value problem
y = f (t, y, y )
y(a) =
y(b) =
(2) Set up finite differences for y and y with step size h:

y (tk )
y (tk )
y(tk+1 ) y(tk1 )
+ O(h2 )
2h
y(tk+1 ) 2y(tk ) + y(tk1 )
+ O(h2 )
h2
(3) Discretize the problem into n = ba

intervals and substitute y(tk ) in the fih
nite difference formulas by wk . Solve the resulting system of linear equations
in w0 , . . . , wn .
47
2.8 Applications
2.8.1 Biology
Example 2.23 (Predator-Prey model)
The Lotka-Volterra predator-prey model describes the relation between two populations. One
population behaves as predator the other as prey. The model consists of a two-dimensional
system of first-order differential equations. The 4 parameters are a, b, c, d > 0.
x = ax + bxy
y = cy + dxy
48
Example 2.24 (Competing population model)
The competing population model describes the two populations that compete in their living
environment. The model consists of a two-dimensional system of first-order differential equations.
The parameter r > 0 describes the effect of the growth of one species on the growth of the other
species.
x = x x2 rxy
y = y y 2 rxy
First, we discuss the case r = 0.5:
x-nullclines: x = 0 and y = 2x + 2
y-nullclines: y = 0 and y = 12 x + 1
Equilibrium solution: (0, 0), (1, 0), (0, 1) and ( 32 , 23 ) (stable)
1.5
0.5
0.5
1.5
Second, we discuss the case r = 2:

x-nullclines: x = 0 and y = 21 x +
1
2
y-nullclines: y = 0 and y = 2x + 1
Equilibrium solution: (0, 0), (1, 0), (0, 1) and ( 31 , 13 ) (unstable)
1
0.8
0.6
0.4
0.2
0
0.2
0.2
0.2
0.4
0.6
0.8
1.2
49
2.8.2 Mechanics
Example 2.25 (Spring damper oscillator)
Let x denote the deflection of a ball with mass m
and c and k denote the spring and damper coefficient,
respectively. Let F (t) be an external force. Newtons
law on the balance of forces states:
F(t)
c
m
m
x + kx + cx = F (t) .
x(t)
Using the new states

y1 = x,
y2 = x
the second-order differential equation can be transformed into a linear first-order system:
0
y 1
=
c
y 2
m
1
0
y
1 + 1

F (t)
y2
m
m
Example 2.26 (Coupled oscillator)

Let x1 and x2 denote the deflection of two balls with
masses m1 and m2 and c1 , c2 , and c3 denote the
spring coefficients. According to Newtons law, we
can model the system with two differential equations:
m1 x
1 + x1 c1 + (x1 x2 )c2
m2 x
2 + x2 c3 + (x2 x1 )c2
=
=
c1
x1
y2 = x2 ,
y3 = x 1 ,
c2
m2
c3
0
0
Using new states

y1 = x1 ,
m1
x2
y4 = x 2
this second-order system can be written as a linear first-order system with constant coefficients:
0

0
y
2
= c +c

2
y 3 1

m
1
y 4
c2
m2
c2
m1
c2 + c3
m2
0
0
y1
y2
y3
y4
50
2.8.3 Engineering
Example 2.27 (Liquid tank)
The model describes the height of a liquid in a spherical tank. The model consists of a first-order
differential equation. The parameter r > 0 describes the tank radius. The parameter cd denotes
a liquid dependent parameter (water: 0.6). The parameter A describes the hole area.
cd A 2gh
h =
2rh h2
For example we can set r = 1.5 m, cd = 0.6, and A = 0.0152 m2 .
3
2.5
h [m]
1.5
0.5
0
0
500
1000
1500
2000
2500
time [s]
51
2.8.4 Vehicle Dynamics

Example 2.28 (Quarter car model with one mass)
The model describes a simple quarter car with one mass:
Source: Mitschke
The model consists of a second-order linear differential equation. The state variables are
y=(
z
) .
z
The model oriented differential equation reads

m
z + kz + cz = kh + ch .
The numerical solver oriented differential equation reads
y = (
0
c
m
1
k )y + (
m
c
m
0
)
k
h+ m
h
52
Example 2.29 (Quarter car model with three masses)
The model describes a simple quarter car with three coupled masses:
Source: Mitschke
The model consists of a system of three second-order linear differential equations. The differential
equations are
m1 z1 k2 (z2 z1 ) c2 (z2 z1 ) + c1 z1
m2 z2 k3 (z3 z2 ) c3 (z3 z2 ) + k2 (z2 z1 ) + c2 (z2 z1 )
m3 z3 + k3 (z3 z2 ) + c3 (z3 z2 )
=
=
=
c1 h
0
0
The state variables are
z1
z = z2
z3
z
y=(
)=
z1
z2
z3
z1
z2
z3
The mass matrix, the right-hand side matrix and the amplification by the road read
m1
M = 0
0
0
m2
0
0
0 ,
m3
c1
R= 0
0
0
0
0
0
0 ,
0
h1
h= 0 .
0
The spring and damper matrices read

k2
K = k2
0
k2
k2 + k3
k3
0
k3 ,
k3
c1 + c2
C = c2
c2
c2 + c3
c3
0
c3 .
c3
Thus, the model oriented differential equation reads

M z + K z + Cz = Rh .
The solver-oriented differential equation reads
y = (
0
M 1 C
E
0
)y + (
) .
M 1 K
M 1 Rh
53
2.9 Exercises
Theoretical Exercises
Exercise 2.1
For each ODE that ist given, determine its order, check if it is linear, and check if the given
functions are solutions.
a) y = 2t with y(t) = t2 + c.
b) y = 2ty + 1 with y(t) = et
x2
dx + et .
e
c) y + 2y 3y = 0 with y1 (t) = e3t and y2 (t) = et .

t
t
d) y + 4y + 3y = t with y1 (t) = and y2 (t) = et + .
3
3
Exercise 2.2
Consider the ODE
y 3y + 2y = t
a) Transform the ODE into a first-order system.

b) Transform the ODE into an autonomous first-order system.
Exercise 2.3
Reformulate each of the following IVPs below into a new IVP which involves a first-order system.
a) y + y et y = sin(3t) with y(0) = 1, y (0) = 2, y (0) = 3.
b) x = xy + t2 + 1 and y = x cos y with x(10) = 4, x (10) = 1 and y(10) = 1.

Exercise 2.4
Suppose we had three different methods M1, M2, and M3 for solving IPVs which had orders 1,
2, and 4, respectively. Suppose also that it is known that for a certain IVP, the constant c of the
global error bound
y(tk ) wk chp ,
could be taken to be 2 for all three methods.

a) Find the resulting error bounds for each of the three methods using a step size h = 0.1.
b) Find the resulting error bounds for each of the three methods using a step size h = 0.05.
c) Compare the results.
54
Exercise 2.5
Compute the region of stability for the trapezoid method
1
1
wk+1 = wk + h ( f (tk , wk ) + f (tk+1 , wk+1 ))
2
2
Computational Exercises
Exercise 2.6
Consider the IVP
y = y,
y(t0 ) = y0 ,
y (t0 ) = y0
y(t0 ) = y0 ,
y(t1 ) = y1 .
and the BVP

y = y,
a) Find the general solution of the differential equation.

b) MATLAB: Set up the IVP solution and draw a graph for t [0, 10].
c) MATLAB: Check if a BVP solution exists. In that case draw a graph for t [0, 10].
Exercise 2.7
MATLAB: Plot the vector fields of the following ODEs for the range [5, 5] [5, 5]. Describe
the solution curves.
y
t
sin t
a) y =
b) y =
c) y = 10
t
y
1 10y
Exercise 2.8
MATLAB: Consider the IVP
y = sin(t2 ),
y(0) = 1 .
a) Plot the solution of the IVP for t [0, 5] using the function quad with stepsize 0.01.
b) Which element of the discrete solution vector y(k) gives y(3)?
c) What is the value of y(3)?
55
Exercise 2.9
Consider the simple IVP
y = y,
y(0) = 1
a) Determine the exact solution.

b) MATLAB: Implement Eulers method and integrate the system with h = 0.2 for t [0, 2].
Plot both, the numerical solution and the exact solution. Compare the curves.
c) MATLAB: Decrease the step size h. Does the numerical solution tend to the exact solution?
Exercise 2.10
Consider the test initial value problem
y = 2y,
y(0) = 5,
t [0, 50]
a) Calculate the exact solution.

b) MATLAB: Integrate the IVP with Eulers method using the step sizes h = 1.015 and 0.9.
What can be observed?
c) MATLAB: Integrate the IVP with the classical Runge-Kutta method of order 4 using the step
sizes h = 1.41 and 1.2. What can be observed?
Application-based Exercises
Exercise 2.11
Population model with two competing species: Consider the system
x = x x2
1
xy,
2
y = y y2
1
xy
2
Draw a phase-plane diagram for x 0, y 0 which includes all equilibrium solutions, x- and
y-nullclines along with the exact flow directions on the nullclines and approximate flow directions
in the regions between nullclines.
Exercise 2.12
After a skydiver jumps from an airplane and until the parachute opens, the air resistance is
proportional to v1.5 , and the maximum speed that the skydiver can reach is 130km/h.
a) Set up the differential equation for the vertical velocity. What is the initial condition?
b) MATLAB: Implement the Runge-Kutta method and integrate the system with h = 0.01 for
t [0, 10]. Plot the solution curve and include the curve if there were no air resistance.
c) MATLAB: How many seconds would it take for the skydiver to break a falling speed of
100km/h?
56
Exercise 2.13
The vertical dynamical behavior of a vehicle can be studied using a quarter car with three masses.
this simplified model consists of a wheel with mass m1 , the body with mass m2 , and the driver
with mass m3 . The masses are linked with springs and dampers.
a) Set up three scalar differential equations of order 2 describing the vertical dynamics using the
positions z1 , z2 , and z3 as state variables. Let h denote the road height.
b) Transform the differential equation into a second-order system with mass matrix M , damper
matrix K, and spring matrix C. Use z = (z1 , z2 , z3 )T as state variable.
c) Transform the system into an explicit first-order system. Use y = (z, z)
T as state variable.
d) MATLAB: Implement the right-hand side function of the ODE system as set up in c).
e) MATLAB: Implement the road height function. Let x denote the road position. Then,
h(x) = 0.2
if x 4
if 4 < x 12
if 12 < x
f) MATLAB: Assume a constant velocity of 10 m

. Furthermore, assume the following masses
s
m1 = 10 , m2 = 100 , m3 = 30
and spring and damping constants
k2 = 1000 , k3 = 100 , c1 = 100000 , c2 = 1000 , c3 = 1000 .
Simulate the quarter car using the built-in solver ode45. Plot the curves z1 (t), z2 (t), z3 (t),
and h(t).
g) MATLAB: Change the parameters mi , ki , and ci and discuss the behavior.
h) MATLAB: Implement the classical RK4 method. Compare the simulation results with this
method and the built-in ode45 method.
i) MATLAB: Animate the model and study the oscillations.
Exercise 2.14
When lighting a match, the ball of flame grows rapidly until it reaches a critical size. Then it
remains at that size because the amount of oxygen being consumed by the combustion in the
interior of the ball balances the amount available through the surface. The simple model with y
representing the radius of the ball is
y = y2 y3 ,
y(0) = ,
2
t [0, ]
a) Show that y(t) = 1 is a steady state solution.

b) MATLAB: Integrate the IVP with the MATLAB routine ode45 using = 0.01. Set the relative
error to 104 .
c) MATLAB: Integrate the IVP with the MATLAB routines ode45 and ode23s using = 0.0001.
Set the relative error to 104 . Compare the number of steps needed. Zoom into the second
half of the graph.
57
3 Partial Differential Equations
Partial differential equations are used as mathematical models in a vast range of technical
applications. Heat distributions, diffusion processes and wave dynamics can be described
with partial differential equations. Any kind of fluid dynamics is mathematically expressed
by the famous Navier-Stokes equations.
58
3.1 Introduction
First of all, we briefly discuss the most important basics on partial differential equations.
3.1.1 Theoretical Basics

The main difference between ordinary and partial differential equations is the number of
independent variables. Solutions of partial differential equations are functions of at least
two variables. Thus, any derivative involved in a partial differential equation is a partial
derivative. Most common partial differential equations are of second-order.
Definition 3.1 (Partial differential equation)

An equation involving a function u(x, y, t) of the independent variables of space x and
y, the independent variable of time t and some partial derivatives of u of highest order
n is called a partial differential equation (PDE) of order n. Special cases involve
functions u(x), u(x, y), and u(x, t).
Example 3.1 (Partial differential equations)
a) The equation ux + uy = 1 is a first-order partial differential equation because two partial
derivatives are involved. The solution is a function u(x, y).
b) The equation ux = t u is also a first-order partial differential equation. Having the independent variables x and t the solution is a function u(x, t).
c) The second-order equation uxx + uyy + uzz = t u2 comprises the space variables x, y, and z,
and time t. Thus, a solution function is of the form u(x, y, z, t).
The independent variables typically describe time and space. It is common to denote
problems in one spacial variable as 1D problems. Likewise, problems in two or three
spacial variables are called 2D and 3D problems.
Dimension
Partial differential equations are called
1D problems if only one spacial variable is involved,
2D problems if two spacial variable are involved,
3D problems if three spacial variable are involved.
This classification holds no matter if time is a further variable or not.
59
Example 3.2 (Problem dimension)
a) The equation ux + uy = 1 is a 2D problem.
b) The equation ux = t u is a 1D partial differential equation although x and time t are both
independent variables.
c) The equation uxx + uyy + uzz = t u2 is a 3D problem.
Example 3.3 (Laplace and Poisson equation)

a) The 1D and 2D Laplace equation reads
u = 0,
= uxx + uyy = 0 .
b) The 1D and 2D Poisson equation reads

u = f (x),
= uxx + uyy = f (x, y) .
Example 3.4 (Heat equation)

With > 0 the 1D and 2D heat equations are defined as
ut = uxx ,
ut = (uxx + uyy ) = u .
Example 3.5 (Wave equation)

The 1D and 2D wave equations are defined as
utt = 2 uxx ,
utt = 2 (uxx + uyy ) .
Example 3.6 (Tricomi equation)

The second-order, linear Tricomi equation is defined as y uxx + uyy = 0.
Solutions of the Laplace equation are called harmonic. Conversely, any function with
u = 0 solves the Laplace equation.
Definition 3.2 (Solution)
A function u is called solution of a partial differential equation if the equation is
satisfied at all points of the domain when inserting u and its partial derivatives.
60
Example 3.7 (Solutions of the heat equation)
a) The function u(x, t) = ax + b solves the heat equation since ut = uxx = 0.
2
b) The function u(x, t) = e t (c1 cos ( x) + c2 sin ( x) ) also solves the heat equation
for any coefficients c1 and c2 since
ut (x, t)
uxx (x, t)
c1 cos ( x) + c2 sin ( x) )
2
e t ( c1 cos ( x) c2 sin ( x) )
2 e t (
2
c) The function u(x, t) = e t (c1 e

any coefficients c1 and c2 .
2
+ c2 e
) also is a solution of the heat equation for
Example 3.8 (Solutions of the Tricomi equation)

The function u(x, y) = y (x2
ux = 2xy,
uxx = 2y,
1
6
y 3 ) solves the Tricomi equation y uxx + uyy = 0 since
uy = x2
2 3
y ,
3
uyy = 2y 2 .
Definition 3.3 (Harmonic function)

A function u is called harmonic if it solves the Laplace equation u = 0.
Example 3.9 (Harmonic functions)
a) The function u(x, y) = x2 + y 2 is not harmonic, since
b) The function u(x, y) = x2 y 2 is harmonic, because
= 2 + 2 = 4 0.
= 2 2 = 0.
c) The function u(x, y) = log(x2 + y 2 ) is harmonic, since

u
2x
2y
2x2 + 2y 2
2x2 2y 2
+
=
+ 2
=0.
2
2
2
2
2
2
2
x x + y
y x + y
(x + y )
(x + y 2 )2
d) Any linear function u(x, y, z) = ax + by + cz + d is harmonic, because all second-order partial

derivatives vanish.
Definition 3.4 (Linear partial differential equation)

An partial differential equation is called linear if u and all its derivaties appear only as
linear terms.
Example 3.10 (Linear and non-linear equations)
a) The equation ux + uy = 1 is linear.
b) The equation ux = t u is also linear.
c) The equation uxx + uyy + uzz = t u2 is non-linear, because u appears as a square.
d) The equation 5ux 7t2 uyy = sin t + et utt is linear. This is not in conflict with the non-linear
coefficients t2 , sin t, or et .
61
Many important partial differential equations are linear and of second order:
a uxx + 2b uxy + c uyy + 2 d1 ux + 2 d2 uy + d3 u = f (x, y).
In general, the coefficients a, b, c and d1 , d2 , d3 depend on (x, y). The factors 2 simplify
some equations related to this quadratic form. Using differentials we can rewrite the
equation as
a b x
d1
u = f .
(
,
)
,
)
(
+
2
(
)
+
d
3
x y
x
y
2
b c
y

A
Finding the zeros of the characteristical polynomial of matrix A
det (A I) = 2 (a + c) + (ac b2 ) = 0
yields the eigenvalues of matrix A:
(a + c) (a + c)2 + 4(b2 ac) (a + c) (a c)2 + 4b2

1,2 =
=
.
2
2
The second expression shows that both eigenvalues are always real numbers. The type
of A follows from the first expression: If b2 ac < 0, then 1,2 > 0 or 1,2 < 0 and A is
positive or negative definite. If b2 ac = 0, then 1 > 0 or 1 < 0 and 2 = 0. Hence, A
is positive or negative semi-definite. Finally, if b2 ac > 0, then 1 > 0 and 2 < 0. This
implies that A is indefinite. So, linear second-order partial differential equations can be
divided into three different types. Note, that in general the type of equation depends of
x and y.
Definition 3.5 (Classification of linear second-order equations)

The linear second-order partial differential equation
a uxx + 2 b uxy + c uyy + 2 d1 ux + 2 d2 uy + d3 u = f (x, y)
is called
elliptic if
b2 ac < 0,
parabolic if
b2 ac = 0,
hyperbolic if
b2 ac > 0.
In general, the coefficients a, b, c and d1 , d2 , d3 depend on (x, y). Hence, the type of
equation depends on (x, y).
62
Example 3.11 (Laplace, heat, and wave equation)
a) For the 2D Laplace and Poisson equation we have
uxx + uyy = 0
a = 1,
b = 0,
c=1
and with that b2 ac = 1 < 0. Hence, the 2D Laplace equation is of second-order, linear,
and elliptic.
b) In terms of classification, the 1D heat equation gives
ut = uxx
a = ,
b = 0,
c=0
which yields b2 ac = 0. Hence, the 1D heat equation is of second-order, linear, and parabolic.
c) For the 1D wave equation we find
utt = 2 uxx
a = 2 ,
b = 0,
c = 1
and with that b2 ac = 2 > 0. Hence, the 1D wave equation is of second-order, linear, and
hyperbolic.
Example 3.12 (Tricomi equation)

Regarding classification the Tricomi equation has the coefficients
y uxx + uyy = 0
a(x, y) = y,
b(x, y) = 0,
c(x, y) = 1 .
This gives b2 ac = y. Hence, the Tricomi equation is elliptic, if y > 0, parabolic if y = 0, and
hyperbolic if y < 0.
Solutions of linear second-order equations

Typical solutions of
elliptic equations are steady-state and do not depend on time.
parabolic equations evolve towards steady-state solutions.
hyperbolic equations do not tend to settle into steady-state.
As with ordinary differential equations partial differential equations typically have infinite
many solutions. To define a well-posed problem with a unique solution, additional initial
and boundary conditions are necessary.
63
Definition 3.6 (Boundary conditions for elliptic equations)

Typical boundary conditions of elliptic partial differential equations with solutions
u(x, y) on a domain D R2 are
y
Dirichlet boundary conditions
u(x, y) = g(x, y),
(x, y) D
Neumann boundary conditions

un (x, y) = g(x, y),
(x, y) D
where un is the partial derivative of u in

normal direction. The special case un = 0 is
called natural boundary conditions.
The solution at P (x0 y0 ) D depends on the whole boundary D.
Example 3.13 (Boundary conditions for the Laplace equation)
a) Consider the 2D Laplace equation on the unit square D = [0, 1] [0, 1]. With
u(0, y) = u(1, y) = 0,
u(x, 0) = 1,
u(x, 1) = 2
Dirichlet boundary conditions are defined.

b) The 2D Laplace equation on the unit circle D = {(x, y) x2 + y 2 1} together with the
Dirichlet boundary condition
u(x, y) = sin(arg(x, y))
for
x2 + y 2 = 1
define a unique solution. Here, arg(x, y) denotes the angle between the vector (x, y)T and
the positive x-axis.
c) Consider the 2D Laplace equation on the triangle D = {(x, y) x, y 0, x + y 1}. With
u(x, 1 x) = 1,
ux (0, y) = uy (x, 0) = 0
mixed Dirichlet and Neumann boundary conditions are defined.
64
Definition 3.7 (Boundary conditions for parabolic equations)

Typical boundary conditions of parabolic partial differential equations with solutions
u(x, t) on a domain D = [a, b] R+0 are
t
D
initial boundary conditions
u(x, 0) = g(x), x [a, b]
P
constant boundary conditions
u(a, t) = c1 , u(b, t) = c2 , t > 0
natural boundary conditions

ux (a, t) = ux (b, t) = 0, t > 0
periodic boundary conditions

u(a, t) = u(b, t), ux (a, t) = ux (b, t), t> 0
The solution at P (x0 t0 ) D depends on that part of the boundary D where t t0 .
Example 3.14 (Boundary conditions for the heat equation)
a) Consider the 1D heat equation on the domain D = [0, 1] R+0 . With
u(x, 0) = sin( x),
u(0, t) = u(1, t),
ux (0, t) = ux (1, t)
initial and periodic boundary conditions are defined.

b) The 2D heat equation on the domain D = {(x, y) x2 + y 2 1} R+0 together with the initial
and constant boundary conditions
u(x, y, 0) = e(x
+y 2 )
u(1, y) = u(1, y) = u(x, 1) = 0,
u(x, 1) = 1
define a unique solution.
Definition 3.8 (Boundary conditions for hyperbolic equations)

Typical boundary conditions of hyperbolic partial differential equations with solutions
u(x, t) on a domain D = [a, b] R+0 are
t
D
initial boundary conditions
u(x, 0) =g(x), ut (x, 0) = g(x), x[a, b]

constant boundary conditions
u(a, t) = c1 ,
u(b, t) = c2 ,
t>0
D
a
The solution at P (x0 t0 ) D depends on that part of the boundary D that lies
behind the propagating line of P .
65
Example 3.15 (Boundary conditions for the wave equation)
Consider the 1D wave equation on the domain D = [0, ] R+0 . With
u(x, 0) = cos x,
u(0, t) = 1,
u(, t) = 1
initial and constant boundary conditions are defined.
3.1.2 Numerical Basics

Some numerical definitions and principles are fundamental and used in the context of any
numerical method for partial differential equations. At least for finite difference approaches
a numerical solution is represented by a finite sequence of grid nodes.
Definition 3.9 (Discretization)

The transformation of a continuous solution into a discrete sequence of points is called
1D-discretization or 2D-discretization in space
u(x)
u(x, y)
ui
ui,j
= u(xi )
= u(xi , yj )
wi
wi,j
1D-discretization or 2D-discretization in space and time

u(x, t)
u(x, y, t)
ui,k
ui,j,k
= u(xi , tk )
= u(xi , yj , tk )
wi,k
wi,j,k
For the continuous solution x [a, b], y [c, d], and t 0. The discretized solution is
defined for i = 0, 1, . . . , n, j = 0, 1, . . . , m, and k N0 . If the points are equidistant with
h > 0 and t > 0 then xi = a + hi, yj = c + hj, and tk = t k. Numerical approximations
of u are denoted by w.
If the problem is 2D or higher, all grid nodes can be ordered linearly in a vector. One of
the possible variants to do this is to use the reading or alphabetical order. All grid nodes
are ordered with ascending index values. The first index has highest and the last index
lowest priority.
66
Definition 3.10 (Reading or alphabetical order)

y
A two-dimensional solution matrix in space with
y2
elements wi,j can be transformed into a onedimensional solution vector w using the reading order or alphabetical order. The elements
y1
wi,j are sorted with ascending index order. The
comparison starts with the first index and in
y0
case of equality continues with the second index.
w0,2 w1,2 w2,2 w3,2

w0,1 w1,1 w2,1 w3,1
w0,0 w1,0 w2,0 w3,0
x0
x1
x2
x3
w = (w0,0 , w0,1 , w0,2 , . . . , w1,0 , w1,1 , w1,2 , . . . , wn,m ) .
Example 3.16 (Reading order)

Consider the domain D = [0, 2] [3, 4]. Using an equally spaced grid with h = 0.1 we obtain the
positions
x0 = 0, x1 = 0.1, x2 = 0.2, . . . , x21 = 2,
y0 = 3, y1 = 3.1, y2 = 3.2, . . . , y11 = 4 .
The grid consists of 21 11 = 231 nodes with 19 9 = 171 inner nodes and 20 + 10 + 20 + 10 = 60
boundary nodes. If the solution function is, for example, w(x, y) = xy 2 , the vector
w = ( 0, 0, 0, . . . , 0.9, 0.961, 1.024, . . . , 1.8, 1.922, 2.048, . . . , . . . , 32)

x=0
x = 0.1
x = 0.2
consists of all grid values in reading order.
67
3.2 Finite Difference Methods

One of the two main streams to solve partial differential equations are finite difference
methods. The idea is to discretize the domain and to approximate all partial derivatives
by finite difference formulas. This is especially simple if an equidistant and orthogonal
grid is used. Then, the resulting linear systems are obtained in a quick and simple way.
This is the main advantage of finite differences. However, if the domain has a curved
boundary or if adaptive grid sizes are used, the difference formulas become much more
complicated.
3.2.1 Finite Differences

Derivatives of functions can easily be approximated by finite differences. Most formulas
result from Taylor expansions. The simple forward difference formula for the first derivative
of a function f is obtained in the following way:
f (t + h) = f (t) + h f (t) + O(h2 )
f (t) =
f (t + h) f (t)
+ O(h) .
h
In the same way, the backward difference formula reads

f (t h) = f (t) h f (t) + O(h2 )
f (t) =
f (t) f (t h)
+ O(h) .
h
Both approximations are of first order O(h). Subtracting the second from the first expanded polynomial
f (t + h)
f (t h)
h2
f (t) +
2
h2
= f (t) h f (t) +
f (t)
2
= f (t) + h f (t) +
yields the central formula for the first derivative

f (t) =
f (t + h) f (t h)
+ O(h2 )
2h
which is of second order O(h2 ).
h3
f (t) + O(h4 )
6
h3
f (t) + O(h4 )
6
68
Definition 3.11 (Finite differences for the first derivative)

The first derivative of a function f R R can be approximated using h > 0 by
the first-order forward and backward difference formula
f (t)
f (t + h) f (t)
,
h
f (t)
f (t) f (t h)
,
h
the second-order central difference formula

f (t)
f (t + h) f (t h)
.
2h
Likewise, adding the expanded polynomials for f (t + h) and f (t h) up to third order

results in the central formula for the second derivative. This central formuls is also of
second order O(h2 ).
Definition 3.12 (Finite differences for the second derivative)
The second derivative of a function f R R can be approximated using h > 0 by
the second-order central difference formula
f (t)
f (t + h) 2f (t) + f (t h)
.
h2
These finite differences in one dimension now can be applied to a function u depending on
x and y. Using these differences, approximations w of u are involved only at grid points.
In this context, we assume a regular grid with equidistant points.
Finite differences
The second-order partial derivatives of a function u(x, y) at point xi , yj can be approximated using h > 0 by the numerical values
uxx (xi , yj )
(wxx )i,j
uyy (xi , yj )
(wyy )i,j
wi+1,j 2wi,j + wi1,j

h2
wi,j+1 2wi,j + wi,j1
h2
69
3.2.2 Difference Methods for 2D Elliptic Equations

We derive a numerical approximation scheme for the 2D Laplace equation. Therefore,
we substitute the second-order partial derivatives of the Laplace equation at each node
(xi , yj )
(uxx + uyy )(xi , yj ) = 0
by finite differences for the second derivative:
wi+1,j 2wi,j + wi1,j wi,j+1 2wi,j + wi,j1
+
=0.
h2
h2
At each node (xi , yj ) the corresponding equation comprises the central value wi,j and
the 4 neighbors wi1,j and wi,j1 . This molecule is called the 5-point-star.
Definition 3.13 (5-point-star)

A numerical solution wi,j of the 2D Laplace
equation can be obtained using the grid size h
and the approximation scheme
wi1,j +wi+1,j +wi,j1 +wi,j+1 4wi,j
=0
h2
which is called the 5-point-star. The coefficients of the 5-point-star molecule are obtained
from the equation multiplied by 1.
y
1
1
1
x
The equations for all numerical values wi,j can be collected in a linear system Aw = b.
If the domain is rectangular, the system matrix A has a special structure, it consists of
5 diagonals. Using a grid with n + 1 nodes in both directions, the vector w consists of
(n + 1)2 elements in reading order and the matrix A has (n + 1)4 elements!
70
Linear system for 5-point-star
Using the 5-point-star to solve the 2D Laplace equation results in the linear system
Aw = b with
4
1
A=
1
1
,
1 4
b=
This linear system is valid for all inner nodes wi,j . Boundary conditions for boundary
nodes have to be added separately. Using the reading order in a retangular grid with
n + 1 nodes in x-direction and m + 1 nodes in y-direction, the values 1 are separated
by m 1 zero elements.
Since all finite differences used in the numerical solution scheme are of order 2, the 5point-star is of second-order. This is sufficient and gives reasonable precision for normal,
well-conditioned problems.
Theorem 3.1 (Order of the 5-point-star)
The numerical method using the 5-point-star with grid size h to solve the 2D Laplace
equation is of order O(h2 ).
Until now, the linear system Aw = b is valid only for inner nodes. Nodes on the boundary
typically have only 2 or 3 neighbors within the domain. Thus, the 5-point-star is not
applicable.
Ghost nodes
Applying the 5-point-star to boundary nodes
generates ghost nodes outside the domain.
These ghost nodes are eliminated again with
modified difference formulas. Thus, for boundary nodes the 5-point-star changes into formulas involving less than 5 nodes.
ghost nodes
ym
y0
x0
xn
To include Dirichlet boundary conditions there are two alternatives. The first alternative is
to keep the boundary nodes in the linear system and replace the corresponding 5-point-star
equations by trivial equations.
71
Boundary nodes with Dirichlet condition
To add Dirichlet boundary conditions, the linear system Aw = b is modified. Any row
k of A that refers to a boundary node wk with Dirichlet conditions is replaced by zeros
and the value 1 on the diagonal. The corresponding element bk of the right-hand side
vector is replaced by the given Dirichlet value gk :
ak,` = 0,
` k,
ak,k = 1,
bk = gk .
Example 3.17 (Laplace equation on a square)

Consider the 2D Laplace problem
u
=0
y
4
on D = {(x, y) 0 x, y 2}
u(x, y) = {
if y = 2
otherwise on D
10
0
The domain is discretized with a grid using h = 1.

First of all, 9 nodes
w0,2
w2,2
w0,0
1
wi,j = w(ih, jh) with i, j = 0, 1, 2
w2,0
2
are introduced.
According to the reading order, we define the vector
w = (w0,0 , w0,1 , w0,2 , w1,0 , w1,1 , w1,2 , w2,0 , w2,1 , w2,2 )T .
Using the 5-point-star we obtain the linear system Aw = b with
A=
1
0
0
0
0
0
0
0
0
0
1
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
4
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
1
0
0
The value of the center node is w1,1 = 2.5.
0
0
0
0
1
0
0
1
0
0
0
0
0
0
0
0
0
1
0
0
10
b= 0 .
10
0
10
The second alternative is to delete all boundary node equations from the linear system. As
an advantage, this implies a smaller matrix A. However, matrix A is then less structured.
This may be regarded as a disadvantage.
72
Example 3.18 (Laplace equation on an L-shaped domain)
y
Consider the 2D Laplace problem
u
=0
on D
u(x, y) = {
5
0
w3,3
if y = 0
otherwise on D
on the given domain D. Using a grid with h = 1 the

domain has 5 inner nodes:
w1,1
w = (w1,1 , w2,1 , w3,1 , w3,2 , w3,3 )T .
w3,1
2
Since the values of the boundary nodes are known through the Dirichlet conditions, we set up a
linear system resulting from the 5-point-star including only the inner nodes:
4w11
4w21
4w31
4w32
4w33
w21
w11
w21
w31
w32
=
=
=
=
=
w31
w32
w33
5
5
5
0
0
4
1
0
0
1
4
1
0
0
0
1
4
1
0
0
0
1
4
1
0
0
0
1
4
w =
5
5
5
0
0
Using the Gau algorithm we get the scheme

4
0
0
0
0
1
15
0
0
0
0
4
56
0
0
0
0
15
209
0
0
0
0
56
780
0
25
100
100
100
and thus the solution values for the inner nodes

w1,1 1.7949,
w2,1 2.1795,
w3,1 1.9231,
w3,2 0.5128,
w3,3 0.1282 .
To include Neumann boundary conditions there are at least two approaches. In a first
approach we could use forward or backward difference formulas for the boundary nodes.
The advantage is that these formulas do not include nodes outside the domain. The big
disadvantage however is that their order is only O(h). This order does not match with
the second-order differences O(h2 ) of the 5-point-star for the inner nodes. A second
approach makes use of temporary ghost nodes outside the domain. If wn,j is a boundary
node on the right part of a boundary, the ghost node wn+1,j lies outside the domain. Let
u
(xn , yj ) = gn,j
n
be the value of the normal derivate at the boundary node. Then, using the central
difference formula in x-direction for the normal derivative gives the approximation
wn+1,j wn1,j
= gn,j
2h
wn+1,j = 2h gn,j + wn1,j .
73
Inserting the ghost node wn+1,j into
4wn,j wn1,j wn+1,j wn,j1 wn,j+1 = 0
finally yields the modified boundary equation
4wn,j 2wn1,j wn,j1 wn,j+1 = 2h gn,j
in which the ghost node is eliminated again.
Boundary nodes with Neumann condition
To add Neumann boundary conditions the linear system Aw = b is modified. In any
row k of A that refers to a boundary node wk with Neumann conditions the element
ak,` that corresponds with the node w` opposite the ghost node outside the domain is
replaced by the value 2. The corresponding element bk of the right-hand side vector
is replaced by a given Neumann value gk and the grid size h:
ak,` = 2,
bk = 2h gk .
As an alternative 9 points can be included in a finite difference scheme. In addition to the

neighbors to the left and right and upwards and downwards, the four diagonal neighbors
can be involved in the approximation formula. A symmetric approach uses the coefficients

The coefficients , , and are chosen such that the resulting local discretisation error
is as small as possible. One possible choice defines the 9-point-star.
Definition 3.14 (9-point-star)

A numerical solution wi,j of the 2D Laplace
equation can be obtained using the grid size h
and the approximation scheme
wi1,j1 + 4wi1,j + 4wi,j1 20wi,j
=0
6h2
which is called the 9-point-star. The coefficients of the 9-point-star molecule are obtained
from the equation multiplied by 1.
y
1
20
1
x
It can be shown that the 9-point-star is of higher order than the 5-point-star. To be more
precise, the order is O(h4 ) in comparison with O(h2 ). However, 9 neighboring points are
74
involved in each equation. The corresponding matrix has more non-zero elements than
the 5-point-star matrix.
Theorem 3.2 (Order of the 9-point-star)
The numerical method using the 9-point-star with grid size h to solve the 2D Laplace
equation is of order O(h4 ).
3.2.3 Difference Methods for 1D Parabolic Equations

The finite difference approach for parabolic equations is quite similar to that for elliptic
ones. For 1D problems the partial derivates with respect to x and t have to substituted
by finite difference formulas.
Definition 3.15 (Explicit 4-point-star)

A numerical solution wi,k of the 1D heat equation ut = uxx can be obtained using the grid
sizes h and t and the approximation scheme
t
1
wi,k+1 wi,k
wi1,k 2 wi,k
=
t
h2
which is called the explicit 4-point-star. The

t
constant is = 2 .
h
12
In contrast to solutions of elliptic problems those of parabolic equations depend on time.

This implies that the numerical solution is computed from one time level to the next one.
Instead of solving one single linear system, an interative process has to be run.
Linear system for explicit 4-point-star
Using the explicit 4-point-star to solve the 1D heat equation ut = uxx results in the
linear system wk+1 = A wk with
1 2

A=
,

1 2
t
.
h2
This linear system is valid for all inner nodes wi,k+1 with 1 i n 1 and k 0.
Boundary conditions for boundary nodes have to be added separately.
75
Example 3.19 (Heat equation on a band domain)
Consider the 1D heat problem
ut = uxx on D = {(x, t) 0 x 4, 0 t}
u(0, t) = 0,
u(4, t) = 10
u(x, 0) = 0
for 0 < x < 4
for 0 t
The domain is discretized with a grid using h = 1 and

1
t = . Using the explicit method, in each interation
2
step
wk+1 = A wk
with
A=
1
1
2
0
0
0
0
0
1
2
0
0
0
1
2
0
1
2
0
0
1
2
0
0
1
1
0
0
0
.
1
2
1
has to be computed. The first and last row in A are due to the Dirichlet boundary conditions.
Starting with the initial condition, the first iterates and the limit interate read
0
0
w0 = 0
0
10
, w1 = 0
10
, w2 = 2.5
10
1.25
2.5
, w3 = 2.5 , . . . , w = 5 .
6.25
7.5
10
10
The simple and straightforward explicit method is of order O(h2 ) and only of order O(t).
Moreover, a stability conditon can be derived. This condition is obtained by investigating
the eigenvalues of matrix A.
Theorem 3.3 (Order and stability of the explicit 4-point-star)
The numerical method using the explicit 4-point-star with grid sizes h and t to solve
the 1D heat equation is of order O(h2 ) and O(t). The method is stable if
=
h2
1
.
2
76
Example 3.20 (Heat equation on a band domain with unstable solution)
Consider again the 1D heat problem
t
ut = uxx on D = {(x, t) 0 x 4, 0 t}
u(0, t) = 0,
u(4, t) = 10
u(x, 0) = 0
for 0 < x < 4
for 0 t
from example 3.19 now with a grid using h = 1 and

t = 1. According to the stability criterion
=
h2
=1>
1
1
1
2
stability is not guaranteed. Using the explicit method, in each interation step
wk+1 = A wk
with
A=
1
1
0
0
0
0
1
1
0
0
0
1
1
1
0
0
0
1
1
0
0
0
0
1
1
has to be computed. The first and last row in A are due to the Dirichlet boundary conditions.
Starting with the initial condition, the first iterates read
0
0
w0 = 0
0
10
0
0
0
0
, w1 = 0 , w2 = 10
10
0
10
10
0
0
10
20
, w3 = 10 , w4 = 40 , . . .
20
20
10
10
This numerical solution shows increasing oscillations and thus do not behave similiar to the real
solution.
Instead of using the forward difference formula for the time discretisation the backward
one can be used as well. The 4-point-star then is established in such a way that just 1
node lies on the old time level and 3 nodes lie on the new one.
Definition 3.16 (Implicit 4-point-star)

wi1,k+1 2wi,k+1
wi,k+1 wi,k
=
t
h2
which is called the implicit 4-point-star. The
t
constant is = 2 .
h
1+2
77
In this implicit approach the matrix times vector multiplication from the explicit method
changes into the solution of a linear system. This implies that the computational effort
increases for the implicit variant.
Linear system for implicit 4-point-star
Using the implicit 4-point-star to solve the 1D heat equation ut = uxx results in the
linear system B wk+1 = wk with
1 + 2

B=
1 + 2
h2
The implicit method is still quite simple and also of order O(h2 ) and only of order O(t).
But in contrast to the explicit approach, the implicit variant is unconditionally stable. This
is a major advantage.
Theorem 3.4 (Order and stability of the implicit 4-point-star)
The numerical method using the implicit 4-point-star with grid sizes h and t to solve
the 1D heat equation is of order O(h2 ) and O(t). The method is unconditionally
stable.
78
Example 3.21 (Heat equation on a band domain with implicit solution)
Consider again the 1D heat problem
t
ut = uxx on D = {(x, t) 0 x 4, 0 t}
u(0, t) = 0,
u(4, t) = 10
u(x, 0) = 0
for 0 < x < 4
for 0 t
from example 3.19 with a grid using h = 1. Using

the implicit method, with t = 1 the corresponding
linear system reads
B wk+1 = wk
with
1
1
B= 0
0
0
0
3
1
0
0
0
1
3
1
0
1
1
0
0
1
3
0
0
0
0
1
1
The first and last row in B are due to the Dirichlet boundary conditions. Starting with the initial
condition, the iterates read
0
0
w0 = 0
0
10
Likewise, with
0
0
0
1.043
2.5
0.476
, w1 1.428 , w2 2.653 , . . . , w = 5 .
3.809
5.487
7.5
10
10
10
1
2
B wk+1 = wk
the corresponding linear system reads
with
11
2
B= 0
0
0
0
2
12
0
0
0
12
2
12
0
0
0
21
2
0
0
0
0
.
1
2
1
Starting with the initial condition, the iterates read

0
0
w0 = 0
0
10
0
0
0
0.178
0.471
2.5
, w1 0.714 , w2 1.530 , . . . , w = 5 .
2.678
4.221
7.5
10
10
10
When making a comparison between the results obtained with the two values of t we have to
be careful. Only every second iterate obtained with t = 21 corresponds to an iterate obtained
with t = 1.
Now, we combine explicit and implicit approach. The two 4-point-stars merge into one
6-point-star called Crank-Nicolson method. This method has been published in a paper
in 1947.
79
Definition 3.17 (6-point-star Crank-Nicolson method)

t
sizes h and t and an approximation scheme
which is the average of the explicit and implicit

4-point-star method and is called the 6-point
star Crank-Nicolson method. The constant
t
is = 2 and the approximation scheme
h
reads
2+2
22
wi,k+1 wi,k
wi1,k + wi1,k+1 2 wi,k 2 wi,k+1
=
.
t
2 h2
Like in the implicit case the Crank-Nicolson method requires the solution of a linear system
in each iteration step. Note, that for 1D parabolic equations the size of the resulting linear
system is comparable to that of 1D elliptic equations and thus in general much smaller
than the size of 2D elliptic equations.
Linear system for 6-point-star Crank-Nicolson method
Using the 6-point-star Crank-Nicolson method to solve the 1D heat equation ut = uxx
t
results in the linear system B wk+1 = A wk with = 2
h
2 2

A=
,

2 2
2 + 2

B=
2 + 2
The Crank-Nicolson method is in several ways satisfying. Firstly, the method is of secondorder for h as well as for t. Secondly, the method is unconditionally stable. There is no
restriction regarding the time step size t.
Theorem 3.5 (Order and stability of the 6-point-star Crank-Nicolson method)
The numerical method using the 6-point-star Crank-Nicolson method with grid sizes h
and t to solve the 1D heat equation is of order O(h2 ) and O(t2 ). The method is
unconditionally stable.
80
Example 3.22 (Heat equation on a band domain with Crank-Nicolson)
u
Consider again the 1D heat problem from example 3.19 with a grid using h = 1. Applying Crank10
Nicolsons method, the corresponding linear system
w
7.5
reads
1
1
0
0
0
0
0
1
0
0
0
1
0
1
0
0
0
1
0
0
0
0
0
1
1
1
1
B= 0
0
0
1
2
2.5
= 1 the corresponding matrices are
A=
w0
B wk+1 = A wk .
With
0
4
1
0
0
0
1
4
1
0
0
0
1
4
0
0
0
0
1
1
The first and last row in B are due to the Dirichlet boundary conditions. Starting with the initial
condition, the iterates read
0
0
w0 = 0
0
10
Likewise, with
A=
0
0
0.357
1.173
2.5
, w1 1.428 , w2 3.265 , . . . , w = 5 .
5.357
6.173
7.5
10
10
10
1
1
2
0
0
0
1
2
0
1
1
2
0
0
the corresponding matrices are

0
1
2
1
1
2
0
0
1
2
1
0
0
0
0
,
1
2
1
11
2
B= 0
0
0
0
3
12
0
0
0
12
3
12
0
0
0
21
3
0
0
0
0
.
1
2
1
Starting with the initial condition, the iterates read

0
0
w0 = 0
0
10
0
0
0.098
0.407
2.5
, w1 0.588 , w2 1.660 , . . . , w = 5 .
3.431
4.852
7.5
10
10
10
When making a comparison between the results obtained with the two values of t we have to
be careful. Only every second iterate obtained with t = 21 corresponds to an iterate obtained
with t = 1.
81
3.2.4 Difference Methods for 2D Parabolic Equations

The extension of difference methods from 1D to 2D in space is straight forward. The
Laplace operator is applied to x and y. This implies the well-known 5-point-star on
the x-y-plane. Like in 1D, there are explicit, implicit, and combined Crank-Nicolson
approaches in 2D as well.
Definition 3.18 (Explicit 6-point-star)

A numerical solution wi,k of the 2D heat equation ut = u can be obtained using the grid
which is called explicit 6-point-star. The constant is
=
t
1
14
h2
and the approximation scheme reads

wi,j,k+1 wi,j,k
wi1,j1,k 4 wi,j,k
=
.
t
h2
The explicit variant result in a molecule with 6 points. The time update is calculated by
the multiplication of a matrix with a vector. Using n + 1 nodes in x-direction and m + 1
nodes in y-direction, the update matrix A consists of (n + 1)2 (m + 1)2 elements.
Linear system for explicit 6-point-star
Using the explicit 6-point-star to solve the 2D heat equation ut = u results in the
linear system wk+1 = A wk with
1 4

A=

1 4
h2
This linear system is valid for all inner nodes wi,j,k+1 with 1 i n1 and 1 j m1
and k 0. Boundary conditions for boundary nodes have to be added separately.
The order of the explicit approach is the same in 1D and 2D. To ensure stability of the
numerical solution, the coefficient has to be below the value 14 . This value in 2D is
different to the value 21 in 1D.
82
Theorem 3.6 (Order and stability of the explicit 6-point-star)

The numerical method using the explicit 6-point-star with grid sizes h and t to solve
the 2D heat equation is of order O(h2 ) and O(t). The method is stable if
=
h2
1
.
4
Swapping the difference formulas of time layers k and k + 1 yields the implicit variant.
Here, the backward difference formula with respect to time is applied. This results again
in a 6-point-star.
Definition 3.19 (Implicit 6-point-star)

A numerical solution wi,k of the 2D heat equation ut = u can be obtained using the grid
which is called implicit 6-point-star. The constant is
=
h2
1+4
and the approximation scheme reads

wi,j,k+1 wi,j,k
wi1,j1,k+1 4 wi,j,k+1
=
.
t
h2
A single node of time k + 1 is computed from several nodes of time k. This again implies
the solution of a linear system. In contrast to elliptic problems, auch a linear system has
to be solved in each time step.
83
Linear system for implicit 6-point-star
Using the implicit 6-point-star to solve the 2D heat equation ut = u results in the
linear system B wk+1 = wk with
1 + 4

B=

1 + 4
t
.
h2
and k 0. Boundary conditions for boundary nodes have to be added separately.
The order of simple explicit and implicit methods is O(h2 ) and O(t). Thus, the implicit
variant has no advantage in terms of precision. But, concerning the overall behavior of
the numerical solution, there is no limit of t. Even with large t, the numerical solution
behaves in the same way as the exact solution.
Theorem 3.7 (Order and stability of the implicit 6-point-star)
The numerical method using the implicit 6-point-star with grid sizes h and t to solve
the 2D heat equation is of order O(h2 ) and O(t). The method is unconditionally
stable.
A further improvement is obtained, using the average of the explicit and the implicit
method. In this approach, 10 points in total are included in the methods molecule. 5
points refer to time k and another 5 points refer to time k + 1.
Definition 3.20 (10-point-star Crank-Nicolson method)
t
A numerical solution wi,k of the 2D heat equa
tion ut = u can be obtained using the grid
2+4
which is the average of the explicit and implicit

6-point-star method and is called the 10-point
star Crank-Nicolson method. The constant
t
24
is = 2 and the approximation scheme
x
reads
wi,j,k+1 wi,j,k
wi1,j1,k + wi1,j1,k+1 2 wi,j,k 2 wi,j,k+1
.
=
t
2 h2
84
The time update involves two matrices. One matrix is multiplied with wk+1 , the other
one with wk . Hence, a matrix multiplication and the solution of a linear system have to
be calculated.
Linear system for 10-point-star Crank-Nicolson method
Using the 10-point-star Crank-Nicolson method to solve the 2D heat equation ut = u
t
results in the linear system B wk+1 = A wk with = 2
h
2 4

A=

2 + 4

, B =

2 4
2 + 4
and k 0. Boundary conditions for boundary nodes have to be added separately. Using
the reading order in a retangular grid with n + 1 nodes in x-direction and m + 1 nodes
in y-direction the values and are separated by m 1 zero elements.
It can be shown, that the order of the Crank-Nicolson approach is 2 for h as well as for
t. Moreover, the advantage of unconditional stbility of the implicit method is inherited
to the Crank-Nicolson method. These advantages make the Crank-Nicolson approach to
a commonly used solver for parabolic problems.
Theorem 3.8 (Order and stability of the 10-point-star Crank-Nicolson method)
The numerical method using the 10-point-star Crank-Nicolson method with grid sizes
h and t to solve the 2D heat equation is of order O(h2 ) and O(t2 ). The method
is unconditionally stable.
3.2.5 Difference Methods for 1D Hyperbolic Equations

The third class of linear, second-order partial differential equations deals is called hyperbolic equations. Typical applications include second partial derivatives with respect to
space and time. In 1D, central difference formulas yield the hyperbolic 5-point-star. This
molecule looks similar to the 5-point-star of elliptic problems but has different coefficients.
85
Definition 3.21 (Hyperbolic 5-point-star)

A numerical solution wi,k of the 1D wave equation utt = 2 uxx can be obtained using the grid
t
1
wi,k1 2wi,k
wi1,k 2wi,k
= 2
2
t
h2
which is called the hyperbolic 5-point-star.

2
t
The constant is = 2 2 .
h
22
1
x
In general, the discretization of space and time is different. This implies a coefficient ,
that also depends on the factor of the differential equation. Moreover, three time layers
are included in the hyperbolic 5-point-star.
Linear system for hyperbolic 5-point-star
Using the hyperbolic 5-point-star to solve the 1D wave equation utt = 2 uxx results in
the second-order linear system wk+1 = A wk wk1 with
2 2

A=
,

2 2
= 2
h2
Boundary conditions for boundary nodes have to be added separately. In addition, a
special initial step is necessary for k = 0.
Since the central difference formula is used for time discretization in the hyperbolic 5point-star, this approach is of second order with respect to space and time. The CourantFriedrichs-Levy condition describes the limit of stability.
Theorem 3.9 (Order and stability of the hyperbolic 5-point-star)
The numerical method using the hyperbolic 5-point-star with grid sizes h and t to
solve the 1D wave equation utt = 2 uxx is of order O(h2 ) and O(t2 ). The method
is stable if the Courant-Friedrichs-Levy condition holds:
= 2
h2
1.
Given the initial conditions

u(x, 0) = g(x),
ut (x, 0) = g(x) ,
86
ghost nodes for the seed computation in the first step are used:
wi,1 wi,1
= gi
2 t
wi,1 = wi,1 2 t gi .
Inserting this formula into the hyperbolic 5-point-star yields

wi,1
= 2(1 )wi,0 + (wi+1,0 + wi1,0 ) wi,1

= 2(1 )gi + (gi+1 + gi1 ) (wi,1 2 t gi ) .
Rearranging the expressions gives the special seed formula

wi,1 = (1 )gi +
(gi1 + gi+1 ) + t gi .
2
Initial nodes with hyperbolic 5-point-star

The initial step using the hyperbolic 5-point-star to solve the 1D wave equation
utt = 2 uxx for the initial conditions
u(x, 0) = g(x),
ut (x, 0) = g(x)
can be computed using the central difference formula in t-direction with ghost nodes
wi,1 outside the domain:
wi,1 = (1 )gi +
(gi1 + gi+1 ) + t gi ,
2
1in1.
This method is of order O(t2 ) .

Finally, one can compare the parts of boundary that influence the exact and the numerical
solution. To ensure stability, the numerically used boundary region has to be superset of
the analytical relevant region.
Stability of hyperbolic difference methods
A numerical method using a difference scheme
is stable if the numerical region of influence covers completely the theoretical region of influence.
t
u
w
87
3.3 Finite Element Methods

The first finite element approaches date back to 1943. In the 1950s and 1960s a practical approach has been established by engineers. After then, in the 1960s and 1970s
mathematicians founded a theoretical basis for finite elements.
The overall idea is to split the domain into small pieces, so-called elements. The original
partial differential equation then is converted into a variational problem. The solution is
a linear combination of basis functions in a finite dimensional function subspace. Finally,
this approach yields a linear system.
3.3.1 Meshes, Partitions, and Triangulations

The basis of any finite element approach is the decomposition of the domain. Independent
of the problem dimension meshes and nodes are the terms used.
Definition 3.22 (Mesh and node)

A mesh decomposes a 1D or 2D domain into a set of elements. In 1D the elements
are subintervals and in 2D the elements can be triangles or quadrangles. The mesh
points are also called nodes.
Typically, in one space dimension a domain is an interval. For 1D problems meshes
specializes to partitions.
Definition 3.23 (Partition)

A partition decomposes a 1D domain into a set of subintervals. The result is a special
mesh on the domain.
Example 3.23 (Partition of an interval)
a) The interval [a, b] = [2, 6] can be decomposed into subintervals in many different ways. One
possible choice is
x0 = a = 2,
x1 = 2.5,
x2 = 3.5,
x3 = 5,
x4 = b = 6 .
This partition is not equidistant.

b) The interval [a, b] = [2, 6] can also be decomposed into n equidistant subintervals using
xk = a + h k,
h=
ba
,
n
k = 0, 1, 2, . . . , n
which yields for n = 40 the nodes

x0 = 2,
x1 = 2.1,
x2 = 2.2,
...
, x3 = 2.3,
...
, x4 = 2.4,
...,
x40 = 6 .
88
For 2D problems meshes can be triangulations. The common way of representing a 2D
triangulation is to use two vectors x and y for the mesh nodes a matrix T for the
triangles. In T only the indices of the nodes are stored. Thus, the elements of T are
natural numbers. The order of the triangle vertices can be either ascending or according
to the orientation. The second variant starts with the vertex with the smallest index and
then continues counter-clockwise.
Definition 3.24 (Triangulation)

A triangulation decomposes a 2D domain into
a set of triangles T1 , . . . , Tm . The result is a
mesh on the domain. The triangulation can be
described with two vectors x Rn and y Rn
containing the coordinates of the n nodes and
a matrix T Rn3 containing in each row the
indices of the three nodes of the corresponding
triangle.
Example 3.24 (Triangulation of a square)

We consider a simple square domain with 1 inner and
4 boundary nodes:
x=
1
2
0
0
2
1
1
1
, y = 2 , T =
0
1
2
3
4
5
3
4
5
2
In each row of T the node indices are arranged starting with the smallest index and then continuing counterclockwise.
y
2
n3
n2
T1
n1 T4
T2
T3
n4
n5
2
Example 3.25 (Triangulation of a domain)

We consider a domain including 1 inner and 5 boundary nodes. The triangulation consists of 5 triangles:
1
1
1
2.5
2
1
0
2
1
x=
0 , y = 0 , T =
2.5
0
2
4
0
y
n3
2
3
4
5
5
3
4
5
2
6
In each row of T the node indices are arranged starting with the smallest index and then continuing counterclockwise.
n2
T1
1
n1 T4
T2
T5
T3
n4
n5
n6
4
89
Example 3.26 (Triangulation of a house shaped domain)
y
We consider a domain with 1 inner and 6 boundary
nodes. The triangulation consists of 6 triangles:
x=
2
1
2
3
1
2
3
2.5
1
1.5
1.5
, y = 1.5 , T = 2
3
0.5
0.5
4
0.5
2
3
3
5
4
6
3
4
5
6
6
7
n1
T1 T2
n4
n3
T3
T5
T4
T6
n7
n6
n2
1
n5
1
In each row of T the node indices are arranged in

ascending order.
The process of decomposing a 2D domain is quite different from that for a 1D interval.
Finding good triangulations is an art in some sense. The Delaunay triangulation is such
that the smallest angle in any triangle is as large as possible. This ensures numerical
stability during the calculation of the finite element solution.
Definition 3.25 (Delaunay triangulation)

The Delaunay triangulation of the convex
hull of given nodes is a triangulation where the
smallest angle in any triangle Tk is as large
as possible:
min max
Tk
The Delaunay triangulation has the property

that no other node lies in the circumcircle of
any triangle.
Delaunay
90
Example 3.27 (Delaunay triangulation of a quarter circle)
y
We consider a quarter circle shaped domain with 1
inner and 7 boundary nodes. Hence, 8 x-coordinates
2
and 8 y-coordinates have to be stored.
1
0
y=
1

3
1
0
x = ,
3
The initial triangulation is changed into a Delaunay

triangulation with large angles.
T =
6
1
1
2
1
1
3
7
6
5
3
8
3
4
8
8
6
8
3
5
5
y
2
n8
n7
n6 n1
T1
T2
T3
n5
T5 T6
T4
n2
T7
1
n3
n4
2
In each row of T the node indices are arranged starting with the smallest index and then continuing
counterclockwise. Finite elements are well suited for curved shaped domains. Hence, for a quarter
circle domain, the finite element method is advantageous in comparison with the finite difference
method.
3.3.2 Variational Problems

This section is a litte bit theoretical, but includes the main principle of the finite element
approach. The idea is to replace the original partial differential equation either by the
principle of virtual work or the minimum potential energy functional. In the latter case,
an optimization problem is solved.
Definition 3.26 (Differential operator)

A partial differential equation can be written with a differential operator in the form
Lu = f .
91
Example 3.28 (Poisson equation with differential operator)
a) The 1D Poisson equation u = f (x) can be reformulated with a differential operator:
Lu = f,
L=
d2
.
d x2
b) In 2D, the Poisson equation uxx + uyy = f (x, y) can be written with an operator:
Lu = f,
L = = 2 =
2
2
+
.
x2 y 2
Sometimes, the equation is multiplied by 1, yielding

Lu = f,
L = ,
f = f .
Definition 3.27 (Inner product)

A mapping u, v of two functions u and v into a real number is called an inner product
if the mapping is
symmetric:
u, v = v, u
positive definite:
u, u > 0 for u 0
linear:
1 u1 + 2 u2 , v = 1 u1 , v + 2 u2 , v
The inner product is a symmetric, positive definite bilinear form.
92
Example 3.29 (Integral expressions as inner products)
a) The integral of the product of two functions u and v on a 1D interval [a, b]
u, v =
u(x) v(x) d x
defines an inner product. Symmetry, linearity, and positive definiteness result directly form
the corresponding properties of the integral.
b) Likewise, the integral of the product of two functions u and v on a 2D domain D
u, v = u(x, y) v(x, y) d x d y
D
defines an inner product.

c) Using the first derivatives of two functions u and v on a 1D interval [a, b] the expression
u , v =
u (x) v (x) d x
is an inner product.
d) In the same way, using the gradients of two functions u and v on a 2D domain D
u, v = u(x, y) v(x, y) d x d y
D
defines an inner product. Note that the function within the integral is real-valued.
Laplace operator and inner product

For the negative Laplace operator Lu = u the inner product is
Lu, v = u, v .
For the negative Laplace operator Lu = u the inner product with a function v is defined
as
Lu, v = u, v =
u v dx dy
Using Greens formula the inner product can be written as

u
v ds + u v dx dy = u, v .
Lu, v =
D
D n

0
u
Here, n
denotes the normal derivative of u. The last expression shows that it is useful
to start with the negative Laplace operator instead of the Laplace operator itself.
93
Definition 3.28 (Properties of a differential operator)

A differential operator L is called
symmetric if
Lu, v = u, Lv
positive definite if
Lu, u > 0 for u 0
94
Example 3.30 (Properties of the negative Laplace operator)
a) We consider the negative 1D Laplace operator Lu = u on [a, b] together with the integral
inner product for functions that are zero on the interval bounds a and b. First, we show the
symmetry of the Laplace operator: Using integration by parts we get
b
Lu, v =
u (x) v(x) d x = u (x) v(x) +

a
b
u (x) v (x) d x .
Since v(a) = v(b) = 0, the first expression vanishes. Likewise, we obtain

b
u, Lv =
u(x) v (x) d x = u(x) v (x) +
u (x) v (x) d x .
Here again, the first expression vanishes. Second, we proof positive definiteness:
b
Lu, u =
u (x) u(x) d x = u (x) u(x) +

a
b
(u (x))2 d x > 0
for any function u 0 with u(a) = u(b) = 0. From this, we can understand that for positive
definiteness it is necessary to use the negative of the original Laplace operator.
b) In 2D, the negative Laplace operator is also symmetric and positive definite for functions
that are zero on the boundary D. Starting with
Lu, v =
u(x, y) v(x, y) d x d y
and using Greens formula we obtain

Lu, v =
u
(s) v(s) d s + u(x, y) v(x, y) d x d y .
n
D
u
denotes the derivative with respect to the normal direction n on the boundary of
n
the domain. With v = 0 on D again the first expression vanishes. Likewise, starting with
Here,
u, Lv = u(x, y) v(x, y) d x d y
D
and again using Greens formula yields

u, Lv =
u(s)
v
(s) d s + u(x, y) v(x, y) d x d y .
n
D
With u = 0 on D we finally get symmetry. Positive definiteness can be seen starting with
Lu, u =
u(x, y) u(x, y) d x d y
and once again using Greens formula:

Lu, u =
u
(s) u(s) d s + (u(x, y))2 d x d y > 0 .
n
D
This holds for any function u 0 with u = 0 on D.
One of the key ideas of the finite element approach is to rewrite the original partial
differential equation as a variational or an optimization problem. The strong formulation
95
then changes into a weak formulation.
Theorem 3.10 (Differential equation and variational problems)
For a symmetric, positive definite, and linear differential operator L the following problems are equivalent:
(A) The function u solves the partial differential equation
Lu = f .
(B) The function u satisfies the principle of virtual work
Lu, v = f, v for all functions v .
(C) The function u minimizes the potential energy functional
F (v) =
1
Lv, v f, v
2
The theorem is fundamental. Hence, it is worth to proof the equality between the three
formulations.
a) (A) (B)
Take the inner product on both sides of the equation Lu = f with an arbitrary
function v.
b) (B) (A)
The opposite direction follows immediately with the fundamental lemma of the calculus of variations: If Lu f, v = 0 for any function v then Lu f has to be the
zero function.
c) (B) (C)
Let v be an arbitrary function. Then, with w = v u we get
F (v) = F (u + w) =
1
L(u + w), u + w f, u + w .
2
Applying the linearity of L and the inner product, the functional F (v) reads
F (v) =
1
(Lu, u + Lu, w + Lw, u + Lw, w) (f, u + f, w) .
2
Since L is assumed to be symmetric, the second and third inner product are the
same:
F (v) =
1
1
Lu, u f, u + Lu, w f, w + Lw, w .
2
2

0
0
F (u)
96
The last expression is positive, since L is assumed to be positive definite. The
inequality F (v) F (u) shows the minimum of F for the function u.
d) (C) (B)
Again, let v be an arbitrary function. Then, we consider the one-dimensional function
() = F (u + v) which can be expanded to
() =
1
(Lu, u + Lu, v + Lv, u + 2 Lv, v) (f, u + f, v) .
2
Using the symmetry of L we can again combine the second and third inner product.
Hence, the derivative with respect to is
() = Lu, v + Lv, v f, v .
F has a minimum at = 0. This yields
0 = (0) = Lu, v f, v .
Theorem 3.11 (Variational problems for Poisson equation)

For the Poisson equation the following problems are equivalent:
(A) The function u solves the partial differential equation
u
=f.
(B) The function u satisfies the principle of virtual work

u, v + f, v = 0
for all functions v S .
(C) The function u minimizes the potential energy functional

F (v) =
1
v, v + f, v
2
Using again the negative Laplace operator Lu = u in combination with the function
f = f the main theorem specializes to the following:
(A) The original partial differential equation yields the Poisson equation:
Lu = f
u = f
=f .
(B) The original principle of virtual work can be written as the following equation:
Lu, v = f, v
u, v = f, v
u, v = f, v .
97
(C) The original potential energy functional specializes to
F (v) =
1
1
1
Lv, v f, v = v, v f, v = v, v + f, v .
2
2
2
Hence, for the Laplace or Poisson equation the principle of virtual work and the potential
energy functional can be formulated with gradients.
3.3.3 Function Spaces

According to the strong or weak formulation of the partial differential equation the solution
has to fulfill different conditions of differentiability. It is common to search numerical
approximations of the solution in finite-dimensional subspaces. This results in a wellposed problem.
Function spaces
A solution u with u = 0 on D of a partial differential equation on a domain D can be
searched for in the following function spaces S S S :
The function space with strong differentiability conditions on u
S = {u D R u C 2 (D)} .
The function space with weak differentiability conditions on u
S = {u D R u C(D), u C 1 piecewise on D} .
A finite-dimensional subspace of S that consists of all piecewise linear functions on
a given mesh of the domain D
S = {u D R u C(D), u piecewise linear on the mesh} .
3.3.4 Piecewise Linear Finite Elements

A numerical solution can be represented by a piecewise linear function. On one hand, such
elements are simple to understand and easy to use. On the other, the precision achieved
is limited. Piecewise linear functions are composed of hat functions. These hat functions
form a basis of the finite-dimensional subspace of all piecewise linear functions. Moreover,
the support of hat functions, that is that part of the domain, where the functions are nonzero is quite small.
98
Definition 3.29 (1D hat functions)

The continuous 1D hat functions on the mesh
a = x1 < . . . < xn = b of the interval [a, b]
j (x) = 0
linear
if x = xj
if x = xi , i j
otherwise
j (x)
with 1 j n define a basis of the space of all

piecewise linear functions S on the mesh.
xj
Similar to 1D, in 2D hat functions also have a very small support. They look like a
carneval hat or a pyramid.
Definition 3.30 (2D hat functions)

The continuous 2D hat functions on a mesh
with nodes (x1 , y1 ), . . . , (xn , yn ) of the domain
D
1 if (x, y) = (xj , yj )
j (x, y) = 0 if (x, y) = (xi , yi ), i j
linear otherwise
z
1
j (x, y)
yj
xj
with 1 j n define a basis of the space of all

piecewise linear functions S on the mesh.
3.3.5 Galerkins Method

Addressing the principle of virtual work which is variant (B) in the main theorem, this
approach is called Galerkins method. A linear combination of the hat functions
n
w = cj j
j=1
is used for the unknown function u. Likewise, the arbitrary function v is one after the
other replaced by the hat functions 1 , . . . , n . In this way, n equations are obtained.
99
Definition 3.31 (Galerkins method)

A numerical solution w of the differential equation Lu = f using Galerkins method
can be found by satisfying the principle of virtual work
Lu, v = f, v
on a finite-dimensional function space S . With basis functions 1 , . . . , n satisfying
homogeneous boundary conditions the solution has the form
n
w(x) = cj j (x)
or
w(x, y) = cj j (x, y) .
j=1
j=1
Using u = w and v = j for 1 j n the principle of virtual work then consists of n

equations for the coefficients c1 , . . . , cn .
3.3.6 Rayleigh-Ritzs Method

Addressing the potential energy functional which is variant (C) in the main theorem,
this approach is called Rayleigh-Ritzs method. Again, a linear combination of the hat
functions
n
w = cj j
j=1
is used for the function v. As a necessary condition the gradient of the functional is set
to zero. Like in the Galerkin approach, here also n equations are obtained.
Definition 3.32 (Rayleigh-Ritzs method)

A numerical solution w of the differential equation Lu = f using Rayleigh-Ritzs
method can be found by minimizing the potential energy functional
F (v) =
1
Lv, v f, v
2
on a finite-dimensional function space S . With basis functions 1 , . . . , n satisfying

homogeneous boundary conditions the solution has the form
n
w(x) = cj j (x)
j=1
or
w(x, y) = cj j (x, y) .
j=1
Using v = w the potential energy functional F then is a function F (c1 , . . . , cn ) and

optimal values for the coefficients c1 , . . . , cn are determined by solving F = 0.
Now we focus on Rayleigh-Ritzs approach for the Poisson equation. In the following, we
100
omit the functions arguments. Inserting
n
w = cj j
j=1
w = cj j
j=1
in the functional we obtain

F (w) =
n
n
1 n
cj j , cj j + f , cj j .
2 j=1
j=1
j=1
This functional can be interpreted as a function depending on c1 , . . . , cn :

F (c1 , . . . , cn ) =
n
1 n n
cj ck j , k + cj f, j .
2 j=1 k=1
j=1
The partial derivative with respect to ci then reads
1 n
2 cj i , j + 2ci i , i + f, i .
F (c1 , . . . , cn ) =
ci
2 ji
The first two expressions can be collected in one sum, and then the partial derivative is
set to zero:
n
F (c1 , . . . , cn ) = cj i , j + f, i = 0 .
ci
j=1
The inner products can be interpreted as matrix and vector elements:

ai,j = i , j ,
bi = f, i ,
i, j = 1, . . . , n .
Note, that the matrix defined as above is symmetric.
Definition 3.33 (Stiffness matrix for Poisson equation)

The symmetric stiffness matrix A Rnn for the 1D Poisson equation consists of the
values
ai,j = i (x) j (x) dx,
D
i, j = 1, . . . , n
and for the 2D Poisson equation of the values

ai,j = i (x, y) j (x, y) dx dy,
D
i, j = 1, . . . , n .
The functions 1 , . . . , n are the basis functions on the domain D.
101
Definition 3.34 (Load vector for Poisson equation)

The load vector b Rn for the 1D Poisson equation consists of the values
bi = f (x) i (x) dx,
D
i = 1, . . . , n
and for the 2D Poisson equation of the values

bi = f (x, y) i (x, y) dx dy,
D
i = 1, . . . , n .
The functions 1 , . . . , n are the basis functions and f is the right-hand side of the
Poisson equation on the domain D.
Using the stiffness matrix and the load vector the Rayleigh-Ritz method for the Poisson
equation finally can be written as a linear system: Ac = b.
Theorem 3.12 (Linear system for Poisson equation)
The values of the Rayleight-Ritz coefficients c1 , . . . , cn can be obtained by solving the
linear system Ac = b, where A is the stiffness matrix and b is the load vector.
The next step addresses the question how the stiffness matrix can be computed. Calculating the elements ai,j one after the other in two nested loops is straightforward but not
efficient, because same integrals are computed several times.
Assembly
The stiffness matrix A and the load vector b can be assembled
nodewise: Each value ai,j and bi is computed separately. This approach is timeconsuming since same parts of the integrals are computed several times.
elementwise: The integrals are computed locally on each mesh element Tk . This
approach generates element stiffness matrices Ak and element load vectors bk .
102
The element-wise assembly calculates the integrals separately on each triangle element. Let
Tk be one triangle with nodes
nk 1 ,
nk2 ,
nk1
nk3 .
Tk
Since the hat function j is non-zero only on

those triangles that include node nj , we obtain
a non-zero integral value on Tk only if
nk3
nk2
i, j {k1 , k2 , k3 } .
This means, that for each triangle, 9 integrals contributing to the stiffness matrix have to
be computed. These 9 values define the element stiffness matrix.
Definition 3.35 (Element stiffness matrix for 2D Poisson equation)

A symmetric element stiffness matrix Ak R33 for the 2D Poisson equation using
a triangulation with triangles Tk consists of the values
ai,j =
(k)
Tk
ki (x, y) kj (x, y) dxdy,
i, j = 1, . . . , 3 .
The functions k1 , k2 , and k3 are the basis functions corresponding to the nodes
k1 , k2 , and k3 of triangle Tk .
In the same way, only if i {k1 , k2 , k3 } we obtain a non-zero entry in the load vector.
Thus, 3 integrals contribute to the element load vector.
Definition 3.36 (Element load vector for 2D Poisson equation)

An element load vector bk R3 for the 2D Poisson equation using a triangulation
with triangles Tk consists of the values
(k)
bi
Tk
f (x, y) ki (x, y) dxdy,
i = 1, . . . , 3 .
The functions k1 , k2 , and k3 are the basis functions corresponding to the nodes
k1 , k2 , and k3 of triangle Tk . The function f is the right-hand side of the Poisson
equation.
Now we address to the computation of the element stiffness matrix. We calculate the 9
103
(k)
values ai,j simultaneously. Let

nk 1 = (
xk1
),
yk1
nk2 = (
xk2
),
yk2
nk3 = (
xk 3
)
yk3
be the coordiantes of the three nodes of triangle Tk . On one hand, the numerical solution
w on Tk is a linear combination of the three relevant hat functions:
w(x, y) = ck1 k1 (x, y) + ck2 k2 (x, y) + ck3 k3 (x, y) .
The transposed gradient of w can be written as a matrix vector product:
T w(x, y)
= ck1 T k1 (x, y) + ck2 T k2 (x, y) + ck3 T k3 (x, y)

ck1
= (T k1 (x, y), T k2 (x, y), T k3 (x, y)) ck2 .
ck3
On the other hand, the numerical solution w on Tk is a linear function:

w(x, y) = 1 + 2 x + 3 y .
At the triangle nodes the solution takes the values ck1 , ck2 , and ck3 :
1 xk1 yk1 1 ck1
1 xk2 yk2 2 = ck2 .
1 xk3 yk3 3 ck3

Fk
Since Tk is a triangle, matrix F k is regular and dk = F k 0. The determinant dk
is positive, if the nodes k1 , k2 , k3 are ordered counterclockwise and negative otherwise.
Hence, we can use Cramers rule to state the components 2 and 3 explicitly:
T w(x, y) = (
c
1
y yk3 yk3 yk1 yk1 yk2 k1
2
( k2
) ck2 .
)=
3
dk xk3 xk2 xk1 xk3 xk2 xk1 c
k3

Bk
Now we can compare the two representations of T w. Since ck1 , ck2 , and ck3 are arbitrary
numbers we have equality of the two matrices:
(T k1 (x, y), T k2 (x, y), T k3 (x, y)) = B k .
The gradients of the hat functions are row vectors with constant numbers. This implies
k1
1
Ak = k2 (T k1 , T k2 , T k3 ) 1 dx dy = B Tk B k dk
2
T
k
k3
using the area
1
2
dk of triangle Tk .
104
Element stiffness matrix for 2D Poisson equation with hat functions
The element stiffness matrix Ak R33 on a triangle Tk with vertices
(
xk1
),
yk1
xk 2
),
yk2
xk3
)
yk3
can be computed with

Ak =
1
dk B Tk B k
2
using the area

RRR 1
R
dk = RRRRR 1
RRR 1
R
1
2
dk of the triangle Tk and coordinates matrix B k :
xk1
xk2
xk3
yk1
yk2
yk3
RRR
RRR
RRR ,
RRR
R
Bk =
1
y yk3
( k2
dk xk 3 xk 2
yk3 yk1
xk1 xk3
yk1 yk2
).
xk2 xk1
105
Example 3.31 (Element stiffness matrices for a square domain)
We again consider example 3.24. The determinants for T1 and T2 are
RRR 1
RR
d1 = RRRR 1
RRR
RR 1
1
2
0
1
2
2
RRR
RRR
RRR = 2 ,
RRR
RR
RRR 1
RR
d2 = RRRR 1
RRR
RR 1
1
0
0
1
2
0
RRR
RRR
RRR = 2 ,
RRR
RR
and in the same way we obtain d3 = 2 and d4 = 2. Since all triangle nodes are in counterclockwise
order, all values dk are positive. The matrix consisting of the gradients of triangle T1 reads
B1 =
1 22
(
2 02
21
10
1
12
0
)= (
21
2 2
1
1
1
) .
1
Likewise, the matrix consisting of the gradients of triangle T2 is

B2 =
1 20
(
2 00
01
10
1 2
12
)= (
01
2 0
1
1
1
) .
1
In the same way, for the two other triangles we get

B3 =
1 0
(
2 2
1
1
1
) ,
1
B4 =
1 2
(
0
2
1
1
1
) .
1
When calculating B 4 we have to be careful about the not ascending order of the node indices:
1, 5, 2. This means k1 = 1, k2 = 5, and k3 = 2. Using the formula Ak = 12 dk B Tk B k , in this
example we obtain 4 equal stiffness matrices:
A1 = A2 = A3 = A4 =
2
1
1
2
1
1
1
0
1
0 .
1
Note the special structure of the element stiffness matrices. The diagonal elements are positive.
The sum in each row and column is zero.
106
Example 3.32 (Element stiffness matrices)
RRR 1
RR
d1 = RRRR 1
RRR
RR 1
1
2.5
0
RRR 1
RR
d2 = RRRR 1
RRR
RR 1
RRR
RRR
RRR = 2.5 ,
RRR
RR
1
2
2
1
0
0
1
2
0
RRR
RRR
RRR = 2
RRR
RR
and in the same way we obtain d3 = 2.5, d4 = 3, and d5 = 3. Since all triangle nodes are ordered
counterclockwise, all values dk are positive. The matrix consisting of the gradients of triangle
T1 reads
B1 =
1
22
(
2.5 0 2.5
21
10
12
0
)=(
2.5 1
1
0.4
) .
0.6
0.4
0.4

B2 =
1 20
(
2 00
01
10
12
1
)=(
01
0
0.5
0.5
0.5
) .
0.5
We also show how the matrix with the gradients of triangle T3 is computed:
B3 =
1
00
(
2.5 2.5 0
01
1 2.5
10
0
)=(
01
1
0.4
0.6
0.4
) .
0.4
For the remaining triangle T4 we obtain

B4 =
1
02
(
3 2.5 2.5
21
1 2.5
10
0.667
)=(
2.5 1
0
0.333
0.5
0.333
) ,
0.5
and finally for the last triangle T5

B5 =
1
00
(
3 4 2.5
Using the formula Ak =

A1 =
02
2.5 4
1
2
20
0
)=(
2.5 2.5
0.5
0.667
0.5
0.667
) .
0
dk B Tk B k we obtain the first element stiffness matrix
1.25
1
d1 B T1 B 1 = 0.5
2
0.75
0.5
0.4
0.1
0.75
0.1 .
0.65
The next two element stiffness matrices read

1
A2 = 0.5
0.5
0.5
0.5
0
0.5
0
0.5
0.75
0.65
0.1
0.5
0.1
0.4
0.375
A5 = 0.375
0.375
1.042
0.667
1.25
A3 = 0.75
0.5
and the last two stiffness matrices are

0.667
A4 = 0.333
0.333
0.333
0.542
0.208
0.333
0.208
0.542
0
0.667 .
0.667
In matrices B k and Ak , some numbers are truncated to 3 digits after the decimal point.
107
Example 3.33 (Element stiffness matrices for a house shaped domain)
We again consider example 3.26. The determinants for the 6 triangles are
d1 = 1,
d2 = 1,
d3 = 1,
d4 = 1,
d5 = 1,
d6 = 1.
Since the orientation of the triangle nodes is not the same we obtain positive and negative values.
The absolute value is constant because the triangulation is regular and all triangles have the same
area. The matrices B k read
B1 = B4 = B6 = (
1
0
0
1
1
1
) , B2 = (
1
0
1
1
0
1
) , B3 = B5 = (
1
1
0
1
1
).
0
The equality of some of the matrices B k is reflected in the element stiffness matrices. The first
two different matrices are
A1 = A4 = A6 =
1
0
1
0
1
1
1
1 ,
2
A2 =
1
1
1
2
0
1
2
1
0
1 ,
1
and the third appearing matrix reads

A3 = A5 =
2
1
1
2
1
1
1
0
1
0 .
1
108
Example 3.34 (Element stiffness matrices for a quarter circle domain)
R
RR
RRRR 1 1
RRRR 1 1
3 RRR
1 RRRR
R
RR
R
R
d1 = RR 1 0
2 RRRR = 1 , d2 = RRRR 1 1
3 RRR = 3 1 ,
RRR
R
R
RRR
1 RRRR
1 RRRR
RR 1 0
RR 1 0
and in the same way we obtain
d3 = 2(2 3) , d4 = 1 ,
d5 = 1 ,
d6 =
31,
d7 = 1 .
Since all triangle nodes are in counterclockwise order, all values dk are positive. The matrix
consisting of the gradients of triangle T1 reads
1 21 1 3
32
1 1 3
32
)=(
) ,
B1 = (
10
01
0
1
1
1 00
1
1
31 11 1 3
31
B2 =
(
)=
(
01 10
11
1
31
31
0
1
3
) .
0
In the same way matrix B 3 can be computed:
1
1 3
31
0
B3 =
) .
(
0
31
2(2 3) 1 3
The next two matrices read
B4 = (
1
1
1
0
0
) ,
1
B5 = (
1
1
1
0
0
) ,
1
and the last two matrices are similar to the first two matrices:
B6 =
1
1
(
31
31
0
3
1
) ,
0
1
B7 = (
32
1
3
0
) .
1
For the element stiffness matrices we switch to floating point numbers:

0.5
A1 = 0.366
0.134
0.366
0.768
0.402
0.134
0.402 ,
0.536
1.049
A2 = 0.683
0.366
0.683
0.683
0
0.366
0 .
0.366
0.536
A7 = 0.402
0.134
0.402
0.768
0.366
0.134
0.366 .
0.5
The next element stiffness matrices are

1
A3 = A4 = A5 = 0.5
0.5
0.5
0.5
0
0.5
0
0.5
and the last two element stiffness matrices read

1.049
A6 = 0.366
0.683
0.366
0.366
0
0.683
0 ,
0.683
109
In the same way the element load vector can be calculated. Since the function f of the
Poisson equation in general is nonlinear, the resulting integrals have to be computed using
numerical integration methods. We do not discuss this issue here.
Elementwise assembly for triangulations
If the mesh is a triangulation with m triangles the elementwise assembly can be performed as follows:
(1) Initialize the stiffness matrix A = 0 and the load vector b = 0.
(2) For k = 1, . . . , m: Add element matrix Ak R33 and element vector bk R3 of
triangle Tk with nodes k1 , k2 , and k3 to A and b.

k1 k2
k3
k1
k2
k3
Example 3.35 (Assembly for a square domain)

We once again consider example 3.24 with the element stiffness matrices from example 3.31.
The assembly starts with A = 0. In the first two steps, matrices A1 and A2 are added to A:
2
1
1
1
2
0
0
1
1
0
0
0
1
0
1
0
0
0
0
0
0
0
0
0
0
0
0
4
1
1
2
2
1
0
1
1
0
0
0
2
0
2
0
0
1
0
0
1
0
0
0
0
0
0
Next, matrix A3 is added. Due to the non-ascending node order we have to add A4 carefully:
6
1
2
2
1
1
1
0
0
0
2
0
2
0
0
2
0
0
2
0
1
0
0
0
1
8
2
2
2
2
2
2
0
0
0
2
0
2
0
0
2
0
0
2
0
2
0
0
0
2
The first row of A remembers of the 5-point-star finite difference method:
A=
4
1
1
1
1
1
1
0
0
0
1
0
1
0
0
1
0
0
1
0
1
0
0
0
1
110
Example 3.36 (Assembly)
The assembly starts with A = 0. After adding 5 matrices we obtain the stiffness matrix
4.167
0.833
1.25
A=
1.25
0.833
0.833
1.317
0.1
0
0.583
0
1.25
0.1
1.15
0
0
0
1.25
0
0
1.15
0.1
0
0.833
0.583
0
0.1
1.983
0.667
0
0
0
0
0.667
0.667
111
Example 3.37 (Assembly for a house shaped domain)
The assembly starts with A = 0. In the first two steps, matrices A1 and A2 are added to A:
1
0
1
1
0
2
0
0
0
0
1
1
0
0
0
0
1
1
2
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
2
1
4
1
0
0
0
0
1
1
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
In the next two steps, matrices A3 and A4 are added to A:

2
0
2
1
0
2
0
0
0
0
3
2
0
1
0
0
2
2
5
1
0
0
0
0
0
1
1
0
0
0
0
1
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
2
0
0
3
2
0
1
0
0
2
2
6
1
0
1
0
0
0
1
1
0
0
0
0
1
0
0
2
1
0
0
0
1
0
1
2
0
0
0
0
0
0
0
0
In the last two steps, matrices A5 and A6 are added to A:

2
0
2
1
0
2
0
0
0
0
3
2
0
1
0
0
2
2
8
2
0
2
0
0
0
2
2
0
0
0
0
1
0
0
2
1
0
0
0
2
0
1
3
0
0
0
0
0
0
0
0
1
0
0
3
2
0
1
0
0
2
2
8
2
0
2
0
0
0
2
3
0
0
1
0
1
0
0
2
1
0
0
0
2
0
1
4
1
0
0
0
1
0
1
2
After adding 6 matrices we obtain the stiffness matrix

1
0
A=
0
0
0
0
0
1.5
1
0
0.5
0
0
1
1
4
1
0
1
0
0
0
1
1.5
0
0
0.5
0
0.5
0
0
1
0.5
0
0
0
1
0
0.5
2
0.5
0
0
0
0.5
0
0.5
1
112
Example 3.38 (Assembly for a quarter circle domain)
The assembly starts with A = 0. After adding 7 matrices we obtain the stiffness matrix
A=
4.098
0
0.866
0
1.183
1.183
0
0.866
0
1
0.5
0
0
0
0
0.5
0.866
0.5
1.902
0.402
0.134
0
0
0
1.183
0
0.134
0.366
1.683
0
0
0
0
0
0.402
0.768
0.366
0
0
0
1.183
0
0
0
0
1.683
0.366
0.134
0
0
0
0
0
0.366
0.768
0.402
0.866
0.5
0
0
0
0.134
0.402
1.902
For the Laplace equation the load vector b up to now is just a zero vector. A nonzero
numerical solution is obtained if boundary conditions are added. As in the finite difference
approach, here we also impress the given boundary conditions on the linear system Ac = b.
Boundary nodes with Dirichlet condition using hat functions
To add Dirichlet boundary conditions the linear system Ac = b is modified. Any row
k of A that refers to a boundary node with Dirichlet conditions is replaced by zeros
and the value 1 on the diagonal. The corresponding element bk of the right-hand side
vector is replaced by a given Dirichlet value gk :
ak,` = 0,
` k,
ak,k = 1,
bk = gk .
Example 3.39 (Boundary conditions for a square domain)

We consider the result of example 3.35. Impressing Dirichlet boundary conditions g2 , . . . , g5 on
the boundary nodes n2 , . . . , n5 yields the simple linear system
4
0
0
0
0
1
1
0
0
0
1
0
1
0
0
1
0
0
1
0
1
0
0
0
1
c1
c2
c3
c4
c5
0
g2
g3
g4
g5
Hence, the value of the unknown inner node w1 is just the average of the boundary nodes:
w1 =
1
(g2 + g3 + g4 + g5 ) .
4
113
Example 3.40 (Boundary conditions for a domain)
We consider the result of example 3.36. Impressing Dirichlet boundary conditions g2 , . . . , g6 on
the boundary nodes n2 , . . . , n6 yields the linear system
4.167
0
0.833
1
0
0
0
0
1.25
0
1
0
0
0
1.25
0
0
1
0
0
0.833
0
0
0
1
0
0
0
0
0
0
1
c1
c2
c3
c4
c5
c6
0
g2
g3
g4
g5
g6
Hence, the value of the unknown inner node w1 is a weighted average of the 4 adjacent boundary
nodes. Boundary value g6 is not relevant for w1 .
Now we have discussed the complete procedure of deriving the overall linear system that
results from the finite element approach using Rayleigh-Ritzs method with hat functions
for the Laplace equation.
Ritzs method with hat functions for Laplace equation with Dirichlet boundary
The finite element solution using Rayleigh-Ritzs method with hat functions k on a
mesh with n nodes and m elements for the 2D Laplace equation with Dirichlet boundary
can be computed as follows:
(1) Initialize the stiffness matrix A Rnn and the load vector b Rn to 0.
(2) Compute the element stiffness matrices Ak and add them to A for k = 1, . . . , m.
(3) Impress Dirichlet boundary conditions on A and b.
(4) Solve the linear system Ac = b. The numerical solution is
n
w(x, y) = cj j (x, y) .
j=1
114
3.4 Applications
3.4.1 Heat Distribution
Example 3.41 (Steady-state heat distribution on a square metal plate)
y
Consider the 2D Laplace equation uxx + uyy = 0 on
10
the given domain D with Dirichlet boundary conditions on the outer boundary
8
u(x, 10) = 100, u(x, 0) = u(0, y) = u(10, y) = 0
and Dirichlet boundary conditions on the inner
boundary (hole)
6
4
2
u(x, 7) = 100, u(x, 3) = u(3, y) = u(7, y) = 0.
10 12
The function u(x, y) describes the steady-state temperature distribution on the metal plate. The solution of the elliptic problem can be computed with
finite differences using the 5-point-star.
Example 3.42 (Time-dependent heat distribution on a square metal plate)

y
Consider the 2D heat equation uxx + uyy = 0 on the
10
given domain D with Dirichlet boundary conditions
on the outer boundary
8
u(x, 10, t) = 100
u(x, 0, t) = u(0, y, t) = u(10, y, t) = 0
and Dirichlet boundary conditions on the inner

boundary (hole)
u(x, 7, t) = 100
u(x, 3, t) = u(3, y, t) = u(7, y, t) = 0.
10 12
10 12
The initial conditions are
10
u(x, y, t) = 0, (x, y) D D
The function u(x, y, t) describes the time-dependent

temperature distribution on the metal plate. The solution of the parabolic problem can be computed with
finite differences using the Crank-Nicolson method.
8
6
4
2
115
Example 3.43 (steady-state heat distribution on a round metal plate)
y
Consider the 2D Laplace equation uxx + uyy = 0 on
10
the given domain D with Dirichlet boundary conditions
8
u = 2,
u = cos(2),
(x 5)2 + (y 5)2 = 22
(x 5)2 + (y 5)2 = 52 .
The function u(x, y) describes the steady-state temperature distribution on the metal plate. The solution of the elliptic problem can be computed with
finite elements using Ritzs method with hat functions.
6
4
2
2
10 12
116
3.5 Exercises
Theoretical Exercises
Exercise 3.1
Show that the function
u(x, y) = ex+y arccos(x y)
solves the partial differential equation
ux + uy 2u = 0
Exercise 3.2
Consider Laplaces differential equation
uxx + uyy = 0
a) For which constant c is the function u = x3 + cxy 2 a solution of the differential equation?
b) Show that u = ln (x a)2 + (y b)2 solves the differential equation.

Exercise 3.3
State the type of the following linear second-order PDEs. If the type changes for different
values of the independent variables, indicate precisely how the type varies with the independent
variables.
a) uxy = ux
b) (x2 1)uxx + 2xyuxy + (y 2 1)uyy = xux + yuy
Exercise 3.4
Show that the local truncation error for the Crank-Nicolson method for the heat equation ut =
uxx is O(t2 ) + O(x2 ) provided that the solution is sufficiently differentiable.
117
Computational Exercises
Exercise 3.5
Consider the 2D-Laplace equation
uxx + uyy = 0,
0 x, y 1
with mixed boundary conditions

u(x, 0) = 0,
u(0, y) = 10,
uy (x, 1) = 3,
ux (1, y) = 0 .
a) Set up a grid over the domain using the step size h = 0.25 . Introduce the nodes wi,j
u(xi , yj ) for 0 i, j 4 (including boundary nodes).
b) Discretize the problem using the 5-point stencil for the PDE and the central difference formula
for the Neumann boundaries.
c) Transform the 2D nodes w0,0 to w4,4 into a 1D node vector w using the reading order. Set
up the matrix A R2525 and the vector b R25 for the linear system Aw = b.
Exercise 3.6
Consider the 2D Laplace equation
uxx + uyy = 0,
0 x, y 3
with Dirichlet boundary conditions

u(x, 3) = 100,
u(x, 0) = u(0, y) = u(3, y) = 0 .
a) Discretize the problem using the 5-point-star with the grid size h = 1.
b) Set up a linear system Aw = b with A R44 and b R4 for the 4 inner nodes.
c) Solve the linear system Aw = b.
d) Triangulate the squares in the grid by lines from left down to right up.
e) Solve the problem with finite element approach. Use Ritzs method with 2D hat functions.
f) Compare the finite difference solution with the finite element solution.
118
Exercise 3.7
uxx + uyy = 0,
0 x, y 2
y
2

u(x, 2) = 100, u(x, 0) = u(0, y) = u(2, y) = 0
on the given domain.

1
a) Set up the stiffness matrix and the load vector of the linear system Ac = b using Ritzs method
with 2D hat functions.
b) Solve the linear system Ac = b.
Exercise 3.8
uxx + uyy = 0
y
2

u(x, 2) = 100, u(x, y) = 0, (x, y) D, y 2
on the given domain D.

1
a) Set up the stiffness matrix and the load vector of the linear system Ac = b using Ritzs method
with 2D hat functions.
b) Solve the linear system Ac = b.
119
Application-based Exercises
Exercise 3.9
Consider the 2D-Laplace equation u = uxx + uyy = 0
on the following domain with the Dirichlet boundary
conditions on the outer boundary
y
10
8
u(x, 10) = 100, u(x, 0) = u(0, y) = u(10, y) = 0

and with the Dirichlet boundary conditions on the inner
boundary (hole)
6
4
2
u(x, 7) = 100, u(x, 3) = u(3, y) = u(7, y) = 0 .
10 12
a) Set up the finite difference method for the steady-state heat problem for the Laplace equation
with Dirichlet boundary conditions using a common step size h = 1.
b) MATLAB: Solve the resulting linear system and give a surface plot of the solution.
c) MATLAB: Give a corresponding isotherm plot.
d) Repeat parts a) through c) using the step size h = 0.5.
Exercise 3.10
on the following domain D with the Dirichlet boundary
conditions on the upper boundary
y
10
8
u(x, 8) = 100
and with zero Dirichlet boundary conditions on the rest

of the boundary
u(x, y) = 0,
(x, y) D,
y8
4
2
2
10 12
a) Set up the finite difference method for the steady-state heat problem for the Laplace equation
with Dirichlet boundary conditions using the common step sizes h = 1 and h = 0.5.
b) MATLAB: Solve the resulting linear system and give a surface plot of the solution.
120
Exercise 3.11
on the following domain D with the Dirichlet boundary
conditions on the outer boundary
y
10
8
u(x, 10) = 100, u(x, 0) = u(0, y) = u(10, y) = 0

and with the Dirichlet boundary conditions on the inner
boundary (hole)
6
4
2
u(x, 7) = 100, u(x, 3) = u(3, y) = u(7, y) = 0 .
10 12
The initial conditions are

u(x, y) = 0, (x, y) D D
a) Set up the 2D Crank-Nicolson method for the time-dependent heat problem. Use h = 0.5
and t = 0.1 for t [0, 5].
b) MATLAB: Solve the resulting linear system and give a surface plot animation of the solution.
c) Set up the explicit difference method for the above problem and compare the resulting linear
system of equations with the Crank-Nicolson scheme.
d) MATLAB: Implement the explicit difference method and compute the solution for
and t = 1.5. Compare the solutions.
Exercise 3.12
Consider the 2D-Laplace problem
u
= 0.05
y
10
= uxx + uyy = 0
with the Dirichlet conditions

u=2
u = cos(2)
,
,
(x 5)2 + (y 5)2 = 22
(x 5)2 + (y 5)2 = 52
6
4
2
on the given domain.

2
10 12
a) MATLAB: Generate approximately n approximately equally spaced nodes within the domain.
Choose a method that respects the symmetry of the domain, and therefore use nodes on
circles with radii between the inner and the outer boundary circle.
b) MATLAB: Generate a triangulation of the domain. Introduce a temporary ghost node to
delete triangles outside the domain.
c) MATLAB: Set up the stiffness matrix and the load vector for the given equation. Impress the
boundary conditions in a final step.
d) MATLAB: Solve the problem and plot a graph of the solution.
e) MATLAB: Vary the number of nodes n.
121
A Appendix
A.1 Greek Letters
Capital Letter
Small Letter
Name
Alpha
Beta
Gamma
Delta
,
Epsilon
Zeta
Eta
Theta
Iota
Kappa
Lambda
My
Ny
Xi
Omikron
Pi
, %
Rho
Sigma
Tau
Ypsilon
Phi
Chi
Psi
Omega
123
Bibliography
[Burden] Burden, Faires: Numerical Analysis, Brooks/Cole

[Hairer] Hairer, Nrsett, Wanner: Solving Ordinary Differential Equations I, II, Springer
[Holmes] Holmes: Introduction to Numerical Methods in Differential Equations, Springer
[Huebner] Huebner, Dewhirst, Smith, Byrom: The Finite Element Method for Engineers,
Wiley
[Smith] Smith: Numerical Solution of Partial Differential Equations: Finite Difference
Methods, Oxford University Press
[Stanoyevitch] Stanoyevitch: Introduction to Numerical Ordinary and Partial Differential
Equations Using MATLAB, Wiley
Vehicle Dynamics
[Genta] Genta: Motor Vehicle Dynamics: Modeling and Simulation, World Scientific
[Gillespie] Gillespie: Fundamentals of Vehicle Dynamics, Society of Automotive Engineers,
Warrendale
[Rajamani] Rajamani: Vehicle Dynamics and Control, Springer
Mathematics
[Koch/Stmpfle] Koch, J., Stmpfle, M.: Mathematik fr das Ingenieurstudium, 2nd edition,
Carl Hanser Verlag, Mnchen, 2012
Formularies
[Bartsch] Bartsch, H.-J.: Taschenbuch mathematischer Formeln, Fachbuchverlag Leipzig
im Carl Hanser Verlag, 21. Auflage, 2007
[Bronstein] Bronstein, Semendjajew, Musiol, Mhlig: Taschenbuch der Mathematik, Harri
Deutsch
[Mohr] Mohr, R.: Mathematische Formeln fr das Studium an Fachhochschulen, Carl
Hanser Verlag, Mnchen, 2011
125
Index
Symbols
A-stable 23
m-step method 20
10-point-star 51
4-point-star,
explicit 46
implicit 47
5-point-star 45
hyperbolic 52
6-point-star 48
explicit 49
implicit 50
9-point-star 46
Crank-Nicolson method 48, 51
D
Delaunay triangulation 54
differential operator 55
positive definite 55
symmetric 55
Dirichlet boundary condition 41
discretization 11, 43
Adams-Bashforth methods 21
Adams-Bashforth-Moulton methods 22
Adams-Moulton methods 21
autonomous 9
element load vector 60

element stiffness matrix 60
elliptic 40
equilibrium solution 10
Eulers method 13
explicit 4-point-star 46
explicit 6-point-star 49
explicit form 8
backward difference formula 44

boundary condition 9
constant 42
costant 42
Dirichlet 41
initial 42
natural 41 f.
Neumann 41
periodic 42
boundary conditions 41 f.
boundary value problem 9
C
central difference formula 44
classical Runge-Kutta method 16
consistent 12
constant boundary condition 42
convergent 12
finite difference method 27

flow line 9
forward difference formula 44
G
Galerkins method 57
general Runge-Kutta method 16
global truncation error 12
H
hat function 57
heat equation 41
Heuns method 15
hyperbolic 40
hyperbolic 5-point-star 52
126
I
implicit 4-point-star 47
implicit 6-point-star 50
implicit Eulers method 13
implicit form 8
improved Eulers method 14
increment function 14, 20
initial boundary condition 42
initial condition 8
initial value problem 8
inner product 55
linear 55
symmetric 55
periodic solution 10
phase portrait 10
phase-plane 10
phase-space 10
Poisson equation 41
positive definite differential operator 55
predictor corrector methods 22
R
Rayleigh-Ritzs method 58
reading order 43
region of stability 23
roundoff error 11
Runge-Kutta-Fehlberg method 19
L
Laplace equation 41
line element 9
linear 8, 40, 55
linear m-step method 20
linear multistep method 20
load vector 59
local truncation error 12
M
matrix,
element stiffness 60
stiffness 59
mesh 54
multistep method 20
N
natural boundary condition 41 f.
Neumann boundary condition 41
node 54
nullcline 11
O
one-step method 14
order 12
ordinary differential equation 8
P
parabolic 40
partial differential equation 40
partition 54
periodic boundary condition 42
S
shooting method 26
stable 11
step size control 18
stiff 25
stiffness matrix 59
symmetric 55
symmetric differential operator 55
system of ordinary differential equations 9
T
triangulation 54
Delaunay 54
truncation error 11
U
unstable 11
V
vector,
element load 60
load 59
vector field 9
W
wave equation 42

Numerische Methoden Lecture Notes

Enviado por

Dados do documento

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Numerische Methoden Lecture Notes

Enviado por

Direitos autorais:

Formatos disponíveis

Mathematical Methods

2 Ordinary Differential Equations

2.1.1 Theoretical Basics

Definition 2.1 (Ordinary differential equation)

Definition 2.2 (Implicit and explicit form)

Definition 2.3 (Linear ordinary differential equation)

b) The equation y = y is an explicit, second-order, linear equation with constant coefficients.

can be rewritten in explicit form.

f) The first-order, non-linear equation y + 3 cos y = 8 y 2 cannot be transformed into an explicit

Definition 2.4 (Initial value problem)

Example 2.2 (Initial value problems)

Definition 2.5 (Boundary value problem)

Example 2.3 (Boundary value problems)

y(t) = sin t + cos t .

y(t) = c1 sin t + cos t .

Boundary value problems

Definition 2.6 (Vector field, flow lines, line elements)

consists of line elements that are in the direction

b) The line elements of the differential equation

are orthogonal to the line elements of y = yt . No

Theorem 2.1 (Vector field)

Definition 2.7 (System of ordinary differential equations)

is called a first-order system of ordinary differential equations.

Definition 2.8 (Autonomous system of ordinary differential equations)

x(n) = f (x, x , . . . , x(n1) )

b) The nonlinear, second-order differential equation with initial values

can be written as a nonlinear system:

c) The nonlinear second-order system with initial values

can be transformed into a first-order system

Transformation of a system into an autonomous system

Example 2.7 (Autonomous system)

Definition 2.9 (Equilibrium solution, periodic solution)

Example 2.8 (Types of solutions)

b) Consider the differential equation y = y. The trivial function y(t) = 0 is an equilibrium

Definition 2.10 (Phase-plane, phase-space, phase portrait)

a) The initial value problem

can be transformed into a system:

The solution curve is an ellipse that is oriented counterclockwise:

b) Rewriting the initial value problem

yields the linear system

The solution curve is a spiral that tends to the origin:

Definition 2.11 (Stability of an equilibrium solution)

2.1.2 Numerical Basics

Definition 2.12 (Discretization)

Definition 2.13 (Truncation error, roundoff error)

discretization is called truncation error,

the limited machine precision is called

Definition 2.14 (Local and global truncation error)

A numerical integration method should be at least consistent. This minimum requirement

Definition 2.15 (Consistence, convergence, order)

of order p > 0 if the local truncation error is bounded by h(p+1) :

convergent if the global truncation error tends to zero:

2.2 Eulers Method and Variants

2.2.1 Eulers Method

Definition 2.16 (Eulers method)

for k N0 . The increment function is the

Theorem 2.2 (Eulers method)

2.2.2 Implicit Eulers Method

Definition 2.17 (Implicit Eulers method)

2.2.3 Improved Eulers Method

Definition 2.18 (Improved Eulers method)

2.2.4 Heuns Method

Definition 2.19 (Heuns method)

2.3 Runge-Kutta Methods