W MATH 745 - T N A: Elcome To Opics in Umerical Nalysis

MATH745 Fall 2005 1
WELCOME TO MATH 745

TOPICS IN NUMERICAL ANALYSIS
Instructor: Dr. Bartosz Protas
Department of Mathematics & Statistics
Email: bprotas@mcmaster.ca
Ofce HH 326, Ext. 24116
Course Webpage: http://www.math.mcmaster.ca/
bprotas/MATH745
MATH745 Fall 2005 2
INTRODUCTION
What is NUMERICAL ANALYSIS ?
Development of COMPUTATIONAL ALGORITHMS for solutions of
problems in algebra and analysis
Use of methods of MATHEMATICAL ANALYSIS to determine a priori
properties of these algorithms such as:
ACCURACY,
STABILITY,
CONVERGENCE
Application of these algorithms to solve actual problems arising in
practice
MATH745 Fall 2005 3
PART I
Finite Differences
A Review
MATH745 Fall 2005 4
DIFFERENTIATION VIA FINITE DIFFERENCES
BASICS
a
(I)
ASSUMPTIONS :
f : R is a smooth function, i.e. is continuously differentiable
sufciently many times,
the domain = [a, b] is discretized with a uniform grid
{x
1
= a, . . . , x
N
= b}, such that x
j+1
x
j
= h
j
= h (extensions to
nonuniform grids are straightforward)
PROBLEM given the nodal values of the function f, i.e., f
j
= f (x
j
),
j = 1, . . . , N approximate the nodal values of the function derivative
d f
dx
(x
j
) = f
(x
j
), j = 1, . . . , N
a
Details can be found in any standard textbook on elementary numerical analysis, e.g.,
K. Atkinson and W. Han, Elementary Numerical Analysis, Wiley, (2004).
MATH745 Fall 2005 5
BASICS (II)
The simplest approach Derivation of nite difference formulae via
TAYLORSERIES EXPANSIONS
f
j+1
= f
j
+(x
j+1
x
j
) f
j
+
(x
j+1
x
j
)
2
2!
f
j
+
(x
j+1
x
j
)
3
3!
f
j
+. . .
= f
j
+h f
j
+
h
2
2
f
j
+
h
3
6
f
j
+. . .
Rearrange the expansion
f
j
=
f
j+1
f
j
h

h
2
f
j
+ =
f
j+1
f
j
h
+O(h),
where O(h
) denotes the contribution from all terms with powers of h

greater or equal (here = 1).
Neglecting O(h), we obtain a FIRST ORDER FORWARDDIFFERENCE
FORMULA :
_
f
x
_
j
=
f
j+1
f
j
h
MATH745 Fall 2005 6
BASICS (III)
Backward difference formula is obtained by expanding f
j1
about x
j
and
proceeding as before:
f
j
=
f
j
f
j1
h

h
2
f
j
+. . . =
_
f
x
_
j
=
f
j
f
j1
h
Neglected term with the lowest power of h is the LEADINGORDER
APPROXIMATION ERROR , i.e., Err =
(x
j
)
f
x
Ch
The exponent of h in the leadingorder error represents the ORDER OF

ACCURACY OF THE METHOD it tells how quickly the approximation
error vanishes when the resolution is rened
The actual value of the approximation error depends on the constant C
characterizing the function f
In the examples above Err =
h
2
f
j
, hence the methods are FIRSTORDER
ACCURATE
MATH745 Fall 2005 7
HIGHERORDER FORMULA (I)
Consider two expansions:
f
j+1
= f
j
+h f
j
+
h
2
2
f
j
+
h
3
6
f
j
+. . .
f
j1
= f
j
h f
j
+
h
2
2
f
j

h
3
6
f
j
+. . .
Subtracting the second from the rst:
f
j+1
f
j1
= 2h f
j
+
h
3
3
f
j
+. . .
Central Difference Formula
f
j
=
f
j+1
f
j1
h

h
2
6
f
j
+. . . =
_
f
x
_
j
=
f
j+1
f
j1
2h
The leadingorder error is
h
2
6
f
j
, thus the method is SECONDORDER
ACCURATE
Manipulating four different Taylor series expansions one can obtain a
fourthorder central difference formula :
_
f
x
_
j
=
f
j+2
+8 f
j+1
8 f
j1
+ f
j2
12h
, Err =
h
4
30
f
(v)
MATH745 Fall 2005 8
APPROXIMATION OF THE SECOND DERIVATIVE
Consider two expansions:
f
j+1
= f
j
+h f
j
+
h
2
2
f
j
+
h
3
6
f
j
+. . .
f
j1
= f
j
h f
j
+
h
2
2
f
j

h
3
6
f
j
+. . .
Adding the two expansions
f
j+1
+ f
j1
= 2 f
j
+h
2
f
j
+
h
4
12
f
iv
j
+. . .
Central difference formula for the second derivative:
f
j
=
f
j+1
2 f
j
+ f
j1
h
2

h
2
12
f
iv
j
+. . . =
_
2
f
x
2
_
j
=
f
j+1
2 f
j
+ f
j1
h
2
The leadingorder error is
h
2
12
f
iv
j
, thus the method is SECONDORDER
ACCURATE
MATH745 Fall 2005 9
AN ALTERNATIVE APPROACH (I)
An alternative derivation of a nitedifference scheme:
Find an Nth order accurate interpolating function p(x) which
interpolates the function f (x) at the nodes x
j
, j = 1, . . . , N, i.e., such that
p(x
j
) = f (x
j
), j = 1, . . . , N
Differentiate the interpolating function p(x) and evaluate at the nodes to
obtain an approximation of the derivative p
(x
j
) f
(x
j
), j = 1, . . . , N
Example:
for j = 2, . . . , N1, let the interpolant have the form of a quadratic
polynomial p
j
(x) on [x
j1
, x
j+1
] (Lagrange interpolating polynomial)
p
j
(x) =
(x x
j
)(x x
j+1
)
2h
2
f
j1
+
(x x
j1
)(x x
j+1
)
h
2
f
j
+
(x x
j1
)(x x
j
)
2h
2
f
j+1
p
j
(x) =
(2x x
j
x
j+1
)
2h
2
f
j1
+
(2x x
j1
x
j+1
)
h
2
f
j
+
(2x x
j1
x
j
)
2h
2
f
j+1
Evaluating at x = x
j
we obtain f
(x
j
) p
j
(x
j
) =
f
j+1
f
j1
2h
(i.e., secondorder accurate centerdifference formula)
MATH745 Fall 2005 10
AN ALTERNATIVE APPROACH (II)
Generalization to higherorders straightforward
Example:
for j = 3, . . . , N2, one can use a fourthorder polynomial as interpolant
p
j
(x) on [x
j2
, x
j+2
]
Differentiating with respect to x and evaluating at x = x
j
we arrive at the
fourthorder accurate nitedifference formula
_
f
x
_
j
=
f
j+2
+8 f
j+1
8 f
j1
+ f
j2
12h
, Err =
h
4
30
f
(v)
Order of accuracy of the nitedifference formula is one less than the order
of the interpolating polynomial
The set of grid points needed to evaluate a nitedifference formula is called
STENCIL
In general, higherorder formulas have larger stencils
TAYLOR TABLE (I)
A general method for choosing the coefcients of a nite difference formula
to ensure the highest possible order of accuracy
Example: consider a onesided nite difference formula
2
p=0
p
f
j+p
,
where the coefcients
p
, p = 0, 1, 2 are to be determined.
Form an expression for the approximation error
f
p=0
p
f
j+p
=
and expand it about x
j
in the powers of h
TAYLOR TABLE (II)
Expansions can be collected in a Taylor table
f
j
f
j
f
j
f
j
f
j
0 1 0 0
a
0
f
j
a
0
0 0 0
a
1
f
j+1
a
1
a
1
h a
1
h
2
2
a
1
h
3
6
a
2
f
j+2
a
2
a
2
(2h) a
2
(2h)
2
2
a
2
(2h)
3
6
the leftmost column contains the terms present in the expression for the
approximation error
the corresponding rows (multiplied by the top row) represent the terms obtained
from expansions about x
j
columns represent terms with the same order in h sums of columns are the
contributions to the approximation error with the given order in h
The coefcients
p
, p = 0, 1, 2 can now be chosen to cancel the
contributions to the approximation error with the lowest powers of h
TAYLOR TABLE (III)
Setting the coefcients of the rst three terms to zero:
_
_
a
0
a
1
a
2
= 0
1a
1
ha
2
(2h) = 0
a
1
h
2
2
a
2
(2h)
2
2
= 0
= a
0
=
3
2h
, a
1
=
2
h
, a
2
=
1
2h
The resulting formula:
_
f
x
_
j
=
f
j+2
+4 f
j+1
3 f
j
2h
The approximation error determined the evaluating the rst column with
nonzero coefcient:
_
a
1
h
3
6
a
2
(2h)
3
6
_
f
j
=
h
2
3
f
j
The formula is thus SECONDORDER ACCURATE
AN OPERATOR PERSPECTIVE (I)
Quick review of FUNCTIONAL ANALYSIS background
NORMED SPACES X: : X R such that x, y X
x 0,
x +y x+y,
x = 0 x 0
Banach spaces
vector spaces: nitedimensional (R
N
) vs. innitedimensional (l
p
)
function spaces (on R
N
): Lebesgue spaces L
p
(), Sobolev spaces
W
p,q
()
Hilbert spaces: inner products, orthogonality & projections, bases, etc.
Linear Operators: operator norms, functionals, Riesz Theorem
AN OPERATOR PERSPECTIVE (II)
Assume that f and f
belong to a function space X; DIFFERENTIATION

d
dx
: f f
can then be regarded as a LINEAR OPERATOR

d
dx
: X X
When f and f
are approximated by their nodal values as f = [ f

1
f
2
. . . f
N
]
T
and f
= [ f
1
f
2
. . . f
N
]
T
, then the differential operator
d
dx
can be
approximated by a DIFFERENTIATION MATRIX A R
NN
such that f
=Af
; How can we determine this matrix?
Assume for simplicity that the domain is periodic, i.e., f
0
= f
N
and
f
1
= f
N+1
; then differentiation with the secondorder center difference
formula can be represented as the following matrixvector product
1
.
.
.
f
=
1
h
0
1
2

1
2
1
2
0
.
.
.
.
.
.
.
.
.
0
1
2
1
2

1
2
0
f
1
.
.
.
f
N

AN OPERATOR PERSPECTIVE (III)
Using the fourthorder center difference formula we would obtain a
pentadiagonal system increased order of accuracy entails increased
bandwidth of the differentiation matrix A
A is a TOEPLITZ MATRIX , since is has constant entries along the the
diagonals; in fact, it a also a CIRCULANT MATRIX with entries a
i j
depending only on (i j)(modN)
Note that the matrix A dened above is SINGULAR (has a zero eigenvalue
= 0) Why?
This property is in fact inherited from the original continuous operator
d
dx
which is also singular and has a zero eigenvalue
A singular matrix A does not have an inverse (at least, now in the classical
sense); what can we do to get around this difculty?
AN OPERATOR PERSPECTIVE (IV)
Matrix singularity linearly dependent rows the LHS vector does not
contain enough information to determine UNIQUELY the RHS vector
MATRIX DESINGULARIZATION incorporating additional information into
the matrix, so that its argument (the RHS vector) can be determined uniquely
Example desingularization of the secondorder center difference
differentiation matrix:
in a center difference formula, even and odd nodes are decoupled
knowing f
j
, j = 1, . . . , N and f
1
, one can recover f
j
, j = 3, 5, . . . (i.e., the
odd nodes) only f
2
must also be provided
hence, the zero eigenvalue has multiplicity two
when desingularizing the differentiation matrix one must modify at least
two rows (see, e.g., sing_diff_mat_01.m )
AN OPERATOR PERSPECTIVE (V)
What is WRONG with the differentiation operator?
The differentiation operator
d
dx
is UNBOUNDED !
One usually cannot nd a constant C R independent of f , such that
d
dx
f (x)
X
Cf
X
,
f X
For instance, f (x) = e
ikx
, so that |C| = k for k ...
Unfortunately, nitedimensional emulations of the differentiation operator
(the DIFFERENTIATION MATRICES ) inherit this property
OPERATOR NORM for matrices
A
2
2
= max
x=1
Ax
2
2
= max
x
(Ax, Ax)
(x, x)
= max
x
(x, A
T
Ax)
(x, x)
=
max
(A
T
A) =
2
max
(A)
Thus, the 2norm of a matrix is given by the square root of its largest
SINGULAR VALUE
max
(A)
AN OPERATOR PERSPECTIVE (VI)
As can be rigorously proved in many specic cases, A
2
grows without
bound as N (or, h 0) this is a reection of the unbounded nature of
the underlying dim operator
The loss of precision when solving the system Ax = b is characterized by the
CONDITION NUMBER (with respect to inversion)
p
(A) =A
p
A
1
p
for p = 2,
2
(A) =

max
(A)
min
(A)
when the condition number is large, the matrix is said to be
ILLCONDITIONED solution of the system Ax = b is prone to
roundoff errors
if A is singular,
p
(A) = +
SUBTRACTIVE CANCELLATION ERRORS
SUBTRACTIVE CANCELLATION ERRORS when comparing two numbers
which are almost the same using niteprecision arithmetic , the relative
roundoff error is proportional to the inverse of the difference between the
two numbers
Thus, if the difference between the two numbers is decreased by an order of
magnitude, the relative accuracy with which this difference may be calculated
using niteprecision arithmetic is also decreased by an order of magnitude.
Problems with nite difference formulae when h 0 loss of precision
due to niteprecision arithmetic( SUBTRACTIVE CANCELLATION ), e.g., for
double precision:
1.00000000000123451.01.2e 12 (2.8% error)
1.00000000000012341.01.0e 13 (19.0% error)
. . .
COMPLEX STEP DERIVATIVE
a
Consider the complex extension f (z), where z = x +iy, of f (x) and compute
the complex Taylor series expansion
f (x
j
+ih) = f
j
+ih f
h
2
2
f
j
i
h
3
6
f
j
+O(h
4
)
Take imaginary part and divide by h
f
j
=
( f (x
j
+ih))
h
+
h
2
6
f
j
+O(h
3
) =
_
f
x
_
j
=
( f (x
j
+ih))
h
Note that the scheme is second order accurate where is conservation of
complexity?
The method doesnt suffer from cancellation errors, is easy to implement and
quite useful
a
J. N. Lyness and C. B.Moler, Numerical differentiation of analytical functions, SIAM J. Numer
Anal 4, 202-210, (1967)
PAD
E APPROXIMATION (I)
GENERAL IDEA include in the nitedifference formula not only the
function values , but also the values of the FUNCTION DERIVATIVE at the
adjacent nodes, e.g.:
b
1
f
j1
+ f
j
+b
1
f
j+1
p=1
p
f
j+p
=
Construct the Taylor table using the following expansions:
f
j+1
= f
j
+h f
j
+
h
2
2
f
j
+
h
3
6
f
j
+
h
4
24
f
(iv)
j
+
h
5
120
f
(v)
j
+. . .
f
j+1
= f
j
+h f
j
+
h
2
2
f
j
+
h
3
6
f
(iv)
j
+
h
4
24
f
(v)
j
+. . .
NOTE need an expansion for the derivative and a higher order expansion
for the function (more coefcient to determine)
PAD
E APPROXIMATION (II)
The Taylor table
f
j
f
j
f
j
f
j
f
(iv)
j
f
(v)
j
b
1
f
j1
0 b
1
b
1
(h) b
1
(h)
2
2
b
1
(h)
3
6
b
1
(h)
4
24
f
j
0 1 0 0 0 0
b
1
f
j+1
0 b
1
b
1
h b
1
h
2
2
b
1
h
3
6
b
1
h
4
24
a
1
f
j1
a
1
a
1
(h) a
1
(h)
2
2
a
1
(h)
3
6
a
1
(h)
4
24
a
1
(h)
5
120
a
0
f
j
a
0
0 0 0 0 0
a
1
f
j+1
a
1
a
1
h a
1
h
2
2
a
1
h
3
6
a
1
h
4
24
a
1
h
5
120
The algebraic system:
0 0 1 1 1
1 1 h 0 h
h h h
2
/2 0 h
2
/2
h
2
/2 h
2
/2 h
3
/6 0 h
3
/6
h
3
/6 h
3
/6 h
4
/24 0 h
4
/24
b
1
b
1
a
1
a
0
a
1
0
1
0
0
0
b
1
b
1
a
1
a
0
a
1
1/4
1/4
3/(4h)
0
3/(4h)

PAD
E APPROXIMATION (III)
The Pad e approximation:
1
4
_
f
x
_
j+1
+
_
f
x
_
j
+
1
4
_
f
x
_
j1
=
3
4h
_
f
j+1
f
j1
_
Leadingorder error
h
4
30
f
(v)
j
( FOURTHORDER ACCURATE )
The approximation is NONLOCAL , in that it requires derivatives at the
adjacent nodes which are also unknowns; Thus all derivatives must be
determined at once via the solution of the following algebraic system
_
_
.
.
.
.
.
.
.
.
.
1/4 1 1/4
.
.
.
.
.
.
.
.
.
_
_
_
_
.
.
.
_
f
x
_
j1
_
f
x
_
j
_
f
x
_
j+1
.
.
.
_
_
=
_
_
.
.
.
.
.
.
3
4h
_
f
j+1
f
j1
_
.
.
.
.
.
.
_
_
PAD
E APPROXIMATION (IV)
Closing the system at ENDPOINTS (where neighbors are not available)
use a lowerorder onesided (i.e., forward or backward) nitedifference
formula
The vector of derivatives can thus be obtained via solution of the following
algebraic system
Bf
=
3
2
Af = f
=
3
2
B
1
Af
where
B is a tridiagonal matrix with b
i,i
= 1 and b
i,i1
= b
i,i+1
=
1
4
,
i = 1, . . . , N
A is a secondorder accurate differentiation matrix
MODIFIED WAVENUMBER ANALYSIS (I)
How do nite differences perform at different WAVELENGTHS ?
FiniteDifference formulae applied to THE FOURIER MODE f (x) = e
ikx
with
the (exact) derivative f
(x) = ike
ikx
CentralDifference formula:
_
f
x
_
j
=
f
j+1
f
j1
2h
=
e
ik(x
j
+h)
e
ik(x
j
h)
2h
=
e
ikh
e
ikh
2h
e
ikx
j
=i
sin(hk)
h
f
j
=ik
f
j
,
where the modied wavenumber k
sin(hk)
h
Comparison of the modied wavenumber k
with the actual wavenumber k

shows how numerical differentiation errors affect different Fourier
components of a given function
MODIFIED WAVENUMBER ANALYSIS (II)
Fourth-order central difference formula
f
x
j
=
f
j+2
+8f
j+1
8f
j1
+ f
j2
12h
=
2
3h
e
ikh
e
ikh
f
j
1
12h
e
ik2h
e
ik2h
f
j
= i
4
3h
sin(hk)
1
6h
sin(2hk)
f
j
= ik
f
j
where the modied wavenumber k
4
3h
sin(hk)
1
6h
sin(2hk)
Fourthorder Pad e scheme:

1
4
f
x
j+1
+
f
x
j
+
1
4
f
x
j1
=
3
4h
( f
j+1
f
j1
),
where
f
x
j+1
= ik
e
ikx
j+1
= ik
e
ikh
f
j
and
f
x
j1
= ik
e
ikx
j1
= ik
e
ikh
f
j
.
Thus:
ik
1
4
e
ikh
+1+
1
4
e
ikh
f
j
=
3
4h
e
ikh
e
ikh
f
j
ik
1+
1
2
cos(kh)
f
j
= i
3
2h
sin(hk) f
j
= k
3sin(hk)
2h+hcos(hk)

W MATH 745 - T N A: Elcome To Opics in Umerical Nalysis

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

W MATH 745 - T N A: Elcome To Opics in Umerical Nalysis

Enviado por

Direitos autorais:

Formatos disponíveis

MATH745 Fall 2005 1

WELCOME TO MATH 745

) denotes the contribution from all terms with powers of h

The exponent of h in the leadingorder error represents the ORDER OF

belong to a function space X; DIFFERENTIATION

can then be regarded as a LINEAR OPERATOR

are approximated by their nodal values as f = [ f

MATH745 Fall 2005 16

MATH745 Fall 2005 24

with the actual wavenumber k

Fourthorder Pad e scheme:

Você também pode gostar