1 Orthogonal functions

Function approximation: Fourier, Chebyshev, Lagrange

Orthogonal functions
Fourier Series
Discrete Fourier Series
Fourier Transform: properties
Chebyshev polynomials
Convolution
DFT and FFT
Scope: Understanding where the Fourier Transform comes
from. Moving from the continuous to the discrete world. The
concepts are the basis for pseudospectral methods and the
spectral element approach.
2 Orthogonal functions
Fourier Series: one way to derive them
The Problem
we are trying to approximate a function f(x) by another function g
n
(x)
which consists of a sum over N orthogonal functions (x) weighted by
some coefficients a
n
.
) ( ) ( ) (
0
x a x g x f
N
i
i i N
=
=
3 Orthogonal functions
... and we are looking for optimal functions in a least squares (l
2
) sense ...
... a good choice for the basis functions (x) are orthogonal functions.
What are orthogonal functions? Two functions f and g are said to be
orthogonal in the interval [a,b] if

=
b
a
dx x g x f 0 ) ( ) (
How is this related to the more conceivable concept of orthogonal
vectors? Let us look at the original definition of integrals:
The Problem
{ } ! Min ) ( ) ( ) ( ) (
2 / 1
2
2
=

=

b
a
N N
dx x g x f x g x f
4 Orthogonal functions
Orthogonal Functions
... where x
0
=a and x
N
=b, and x
i
-x
i-1
=x ...
If we interpret f(x
i
) and g(x
i
) as the ith components of an N component
vector, then this sum corresponds directly to a scalar product of vectors.
The vanishing of the scalar product is the condition for orthogonality of
vectors (or functions).

=

N
i
i i
b
a
N
x x g x f dx x g x f
1
) ( ) ( lim ) ( ) (
f
i
g
i
0 = =
i
i
i i i
g f g f
5 Orthogonal functions
Periodic functions
-15 -10 -5 0 5 10 15 20
0
10
20
30
40
Let us assume we have a piecewise continuous function of the form
) ( ) 2 ( x f x f = +
2
) ( ) 2 ( x x f x f = = +
... we want to approximate this function with a linear combination of 2

periodic functions:
) sin( ), cos( ),..., 2 sin( ), 2 cos( ), sin( ), cos( , 1 nx nx x x x x
{ }

=
+ + =
N
k
k k N
kx b kx a a x g x f
1
0
) sin( ) cos(
2
1
) ( ) (
6 Orthogonal functions
Orthogonality
... are these functions orthogonal ?
0 , 0 0 ) sin( ) cos(
0
0 , , 0
) sin( ) sin(
0
0 2
0
) cos( ) cos(
> =
> =
>

=
> =
= =

k j dx kx jx
k j
k j k j
dx kx jx
k j
k j
k j
dx kx jx

... YES, and these relations are valid for any interval of length 2.
Now we know that this is an orthogonal basis, but how can we obtain the
coefficients for the basis functions?
from minimising f(x)-g(x)
7 Orthogonal functions
Fourier coefficients
optimal functions g(x) are given if
{ } 0 ) ( ) ( ! Min ) ( ) (
2 2
= =

x f x g or x f x g
n
a
n
k
leading to
... with the definition of g(x) we get ...
{ }

+ +

=
dx x f kx b kx a a
a
x f x g
a
N
k
k k
k
n
k
2
1
0
2
) ( ) sin( ) cos(
2
1
) ( ) (
2

{ }

=
= =
= =
+ + =

N k dx kx x f b
N k dx kx x f a
kx b kx a a x g
k
k
N
k
k k N
,..., 2 , 1 , ) sin( ) (
1
,..., 1 , 0 , ) cos( ) (
1
with ) sin( ) cos(
2
1
) (
1
0
8 Orthogonal functions
Fourier approximation of |x|
... Example ...
.. and for n<4 g(x) looks like
leads to the Fourier Serie

+ + + = ...
5
) 5 cos(
3
) 3 cos(
1
) cos( 4
2
1
) (
2 2 2
x x x
x g

= x x x f , ) (
-20 -15 -10 -5 0 5 10
0
1
2
3
4
9 Orthogonal functions
Fourier approximation of x
2
... another Example ...
2 0 , ) (
2
< < = x x x f
.. and for N<11, g(x) looks like
leads to the Fourier Serie

+ =
N
k
N
kx
k
kx
k
x g
1
2
2
) sin(
4
) cos(
4
3
4
) (

-10 -5 0 5 10
-10
0
10
20
30
40
10 Orthogonal functions
Fourier - discrete functions
i
N
x
i
2
=
.. the so-defined Fourier polynomial is the unique interpolating function to
the function f(x
j
) with N=2m
it turns out that in this particular case the coefficients are given by
,... 3 , 2 , 1 , ) sin( ) (
2
,... 2 , 1 , 0 , ) cos( ) (
2
1
*
1
*
= =
= =

=
=
k kx x f
N
b
k kx x f
N
a
N
j
j j
N
j
j j
k
k
{ } ) cos(
2
1
) sin( ) cos(
2
1
) (
*
1
1
* * * *
0
kx a kx b kx a a x g
m
m
k
m
k k
+ + + =

=
... what happens if we know our function f(x) only at the points
11 Orthogonal functions
) ( ) (
*
i i m
x f x g =
Fourier - collocation points
... with the important property that ...
... in our previous examples ...
-10 -5 0 5
0
0.5
1
1.5
2
2.5
3
3.5
f(x)=|x| => f(x) - blue ; g(x) - red; x
i
- +
12 Orthogonal functions
Fourier series - convergence
f(x)=x
2
=> f(x) - blue ; g(x) - red; x
i
- +
13 Orthogonal functions
Fourier series - convergence
f(x)=x
2
=> f(x) - blue ; g(x) - red; x
i
- +
14 Orthogonal functions
Gibbs phenomenon
f(x)=x
2
=> f(x) - blue ; g(x) - red; x
i
- +
0 0.5 1 1.5
-6
-4
-2
0
2
4
6
N =32
0 0.5 1 1.5
-6
-4
-2
0
2
4
6
N =16
0 0.5
-6
-4
-2
0
2
4
6
0 0.5 1 1.5
-6
-4
-2
0
2
4
6
N =128
0 0.5 1 1.5
-6
-4
-2
0
2
4
6
N =256
The overshoot for equi-
spaced Fourier
interpolations is 14% of
the step height.
15 Orthogonal functions
Chebyshev polynomials
We have seen that Fourier series are excellent for interpolating
(and differentiating) periodic functions defined on a regularly
spaced grid. In many circumstances physical phenomena which
are not periodic (in space) and occur in a limited area. This quest
leads to the use of Chebyshev polynomials.
We depart by observing that cos(n) can be expressed by a
polynomial in cos():
1 cos 8 cos 8 ) 4 cos(
cos 3 cos 4 ) 3 cos(
1 cos 2 ) 2 cos(
2 4
3
2
+ =
=
=



... which leads us to the definition:
16 Orthogonal functions
Chebyshev polynomials - definition
N n x x x T T n
n n
= = = ], 1 , 1 [ ), cos( ), ( )) (cos( ) cos(
... for the Chebyshev polynomials T
n
(x). Note that because of
x=cos() they are defined in the interval [-1,1] (which - however -
can be extended to ). The first polynomials are
0
2 4
4
3
3
2
2
1
0
and ] 1 , 1 [ for 1 ) (
where 1 8 8 ) (
3 4 ) (
1 2 ) (
) (
1 ) (
N n x x T
x x x T
x x x T
x x T
x x T
x T
n

+ =
=
=
=
=
17 Orthogonal functions
Chebyshev polynomials - Graphical
The first ten polynomials look like [0, -1]
The n-th polynomial has extrema with values 1 or -1 at
0 0.2 0.4 0.6 0.8
-1
-0.5
0
0.5
1
x
T
_
n
(
x
)
n k
n
k
x
ext
k
,..., 3 , 2 , 1 , 0 ), cos(
) (
= =

18 Orthogonal functions
Chebyshev collocation points
These extrema are not equidistant (like the Fourier extrema)
n k
n
k
x
ext
k
,..., 3 , 2 , 1 , 0 ), cos(
) (
= =

k
x(k)
19 Orthogonal functions
Chebyshev polynomials - orthogonality
... are the Chebyshev polynomials orthogonal?
0
2
1
1
, ,
0
0 2 /
0
1
) ( ) ( N j k
j k for
j k for
j k for
x
dx
x T x T
j k

= =
> =

Chebyshev polynomials are an orthogonal set of functions in the

interval [-1,1] with respect to the weight function
such that
2
1 / 1 x
... this can be easily verified noting that
) cos( ) ( ), cos( ) (
sin , cos


j x T k x T
d dx x
j k
= =
= =
20 Orthogonal functions
Chebyshev polynomials - interpolation
... we are now faced with the same problem as with the Fourier
series. We want to approximate a function f(x), this time not a
periodical function but a function which is defined between [-1,1].
We are looking for g
n
(x)
) ( ) (
2
1
) ( ) (
1
0 0
x T c x T c x g x f
n
k
k k n
=
+ =
... and we are faced with the problem, how we can determine the
coefficients c
k
. Again we obtain this by finding the extremum
(minimum)
{ } 0
1
) ( ) (
2
1
1
2
=

x
dx
x f x g
c
n
k
21 Orthogonal functions
Chebyshev polynomials - interpolation
... to obtain ...
n k
x
dx
x T x f c
k k
,..., 2 , 1 , 0 ,
1
) ( ) (
2
1
1
2
=

... surprisingly these coefficients can be calculated with FFT

techniques, noting that
n k d k f c
k
,..., 2 , 1 , 0 , cos ) (cos
2
0
= =

... and the fact that f(cos) is a 2-periodic function ...
n k d k f c
k
,..., 2 , 1 , 0 , cos ) (cos
1
= =

... which means that the coefficients c

k
are the Fourier coefficients
a
k
of the periodic function F()=f(cos )!
22 Orthogonal functions
Chebyshev - discrete functions
i
N
x
i

cos =
... leading to the polynomial ...
in this particular case the coefficients are given by
2 / ,... 2 , 1 , 0 , ) cos( ) (cos
2
1
*
N k k f
N
c
N
j
j j
k
= =

=

=
+ =
m
k
k k m
x T c T c x g
1
* *
0
*
) (
2
1
) (
0
... what happens if we know our function f(x) only at the points
... with the property
N 0,1,2,..., j j/N) cos( x at ) ( ) (
j
*
= = = x f x g
m
23 Orthogonal functions
Chebyshev - collocation points - |x|
f(x)=|x| => f(x) - blue ; g
n
(x) - red; x
i
- +
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
0
0.2
0.4
0.6
0.8
1
N = 8
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
0
0.2
0.4
0.6
0.8
1
N = 16
8 points
16 points
24 Orthogonal functions
Chebyshev - collocation points - |x|
f(x)=|x| => f(x) - blue ; g
n
(x) - red; x
i
- +
32 points
128 points
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
0
0.2
0.4
0.6
0.8
1
N =32
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.
0
0.2
0.4
0.6
0.8
1
N =128
25 Orthogonal functions
Chebyshev - collocation points - x
2
f(x)=x
2
=> f(x) - blue ; g
n
(x) - red; x
i
- +
8 points
64 points
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
0
0.2
0.4
0.6
0.8
1
1.2
N =8
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
1.2
N =64
The interpolating
function g
n
(x) was
shifted by a small
amount to be
visible at all!
26 Orthogonal functions
Chebyshev vs. Fourier - numerical
f(x)=x
2
=> f(x) - blue ; g
N
(x) - red; x
i
- +
This graph speaks for itself ! Gibbs phenomenon with Chebyshev?
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
N =16
0 2
-5
0
5
10
15
20
25
30
35
N =
Chebyshev
Fourier
27 Orthogonal functions
Chebyshev vs. Fourier - Gibbs
f(x)=sign(x-
)
=> f(x) - blue ; g
N
(x) - red; x
i
- +
Gibbs phenomenon with Chebyshev? YES!
Chebyshev
Fourier
-1 -0.5 0 0.5 1
-1.5
-1
-0.5
0
0.5
1
1.5
N =16
0 2
-1.5
-1
-0.5
0
0.5
1
1.5
N =16
28 Orthogonal functions
Chebyshev vs. Fourier - Gibbs
f(x)=sign(x-
)
=> f(x) - blue ; g
N
(x) - red; x
i
- +
Chebyshev
Fourier
-1 -0.5 0 0.5 1
-1.5
-1
-0.5
0
0.5
1
1.5
N =64
0 2 4
-1.5
-1
-0.5
0
0.5
1
1.5
N =6
29 Orthogonal functions
Fourier vs. Chebyshev
Fourier
Chebyshev
i
N
x
i
2
=
i
N
x
i

cos =
periodic functions
limited area [-1,1]
) sin( ), cos( nx nx

cos
), cos( ) (
=
=
x
n x T
n
{ }
) cos(
2
1
) sin( ) cos(
2
1
) (
*
1
1
* *
* *
0
kx a
kx b kx a
a x g
m
m
k
m
k k
+
+ +
=

=
+ =
m
k
k k m
x T c T c x g
1
* *
0
*
) (
2
1
) (
0
collocation points
domain
basis functions
interpolating
function
30 Orthogonal functions
Fourier vs. Chebyshev (contd)
Fourier
Chebyshev
coefficients
some properties

=
=
=
=
N
j
j j
N
j
j j
kx x f
N
b
kx x f
N
a
k
k
1
*
1
*
) sin( ) (
2
) cos( ) (
2

=
=
N
j
j j
k f
N
c
k
1
*
) cos( ) (cos
2

Gibbs phenomenon for
discontinuous functions
Efficient calculation via FFT
infinite domain through
periodicity
limited area calculations
grid densification at boundaries
coefficients via FFT
excellent convergence at
boundaries
Gibbs phenomenon
31 Orthogonal functions
The Fourier Transform Pair

=
=

d e F t f
dt e t f F
t i
t i
) ( ) (
) (
2
1
) (
Forward transform
Inverse transform
Note the conventions concerning the sign of the exponents and the factor.
32 Orthogonal functions
Some properties of the Fourier Transform
Defining as the FT:
) ( ) ( F t f
Linearity
Symmetry
Time shifting
Time differentiation
) ( ) ( ) ( ) (
2 1 2 1
bF aF t bf t af + +
) ( 2 ) ( F t f
) ( ) (

F e t t f
t i
+
) ( ) (
) (
F i
t
t f
n
n
n

33 Orthogonal functions
Differentiation theorem
Time differentiation ) ( ) (
) (
F i
t
t f
n
n
n

34 Orthogonal functions
Convolution

= = ' ) ' ( ) ' ( ' ) ' ( ) ' ( ) ( ) ( dt t g t t f dt t t g t f t g t f
The convolution operation is at the heart of linear systems.
Definition:
Properties:
) ( ) ( ) ( ) ( t f t g t g t f =
) ( ) ( ) ( t f t t f =

= dt t f t H t f ) ( ) ( ) (
H(t) is the Heaviside function:
35 Orthogonal functions
The convolution theorem
A convolution in the time domain corresponds to a
multiplication in the frequency domain.
and vice versa
a convolution in the frequency domain corresponds to a
multiplication in the time domain
) ( ) ( ) ( ) ( G F t g t f
) ( ) ( ) ( ) ( G F t g t f
The first relation is of tremendous practical implication!
36 Orthogonal functions
Summary
The Fourier Transformcan be derived fromthe problemof
approximating an arbitrary function.
A regular set of points allows exact interpolation (or derivation) of
arbitrary functions
There are other basis functions (e.g., Chebyshev polynomials,
Legendre polynomials) with similar properties
These properties are the basis for the success of the spectral
element method