Math 572

Fourier Series, Discrete Fourier Transforms and Fast Fourier Transforms

1. Complex Numbers
We start with some basics of complex numbers:
If z = a + bi is a complex number, then the complex conjugate of z is the complex number z = a bi.
From Eulers theorem, we have a relationship between e
x
, cos x and sin x:
e
ix
= cos x + i sin x
and
e
ix
= cos x i sin x
We will dene
N
= e

2i
N
for N a positive integer.
N
is called a primitive Nth root of unity. In
particular,
N
is a root of x
N
1 and is not a root of x
k
1 for k < N. Some useful properties of
N
are:

k
N
=
M
if N = kM

N/2
N
= 1 if N is even

M
N
=
r
N
if M = kN + r
2. Fourier Series over [0, 2]
In this section we consider the function space C[0, 2]. Our initial goal will be to approximate a given function
f(x) in C[0, 2] using linear combinations of functions of the form cos kx and sin kx where k is a non-negative
integer. This is obviously useful for periodic functions. Our setup is as follows:
f(x), g(x) =
1

_
2
0
f(x)g(x) dx
and
S
n
= Span(
1

2
, sin x, . . . , sin nx, cos x, . . . , cos nx)
We can check that the following are true:
1, cos x =
1

_
2
0
cos x dx = 0
1, sin x =
1

_
2
0
sin x dx = 0
cos x, cos x =
1

_
2
0
cos xcos x dx =
_
_
_
1 if =
0 if =
sin x, sin x =
1

_
2
0
sin xsin x dx =
_
_
_
1 if =
0 if =
cos x, sin x =
1

_
2
0
cos xsin x dx = 0
Therefore
{
1

2
, sin x, . . . , sin kx, cos x, . . . , cos kx}
is an orthonormal basis for S
k
.
1
If we dene:
a
0
= f(x), 1 =
1

_
2
0
f(x) dx
a
k
= f(x), cos kx =
1

_
2
0
f(x) cos kx dx
and
b
k
= f(x), sin kx =
1

_
2
0
f(x) sin kx dx
then the least squares approximation of f(x) in S
n
will be:
s
n
(f(x)) =
a
0
2
+
n

k=1
(a
k
cos kx + b
k
sin kx)
This sum is an example of a trigonometric polynomial and is known as the nth degree Fourier series for
f(x) over [0, 2] .
It is generally convenient to write this series in exponential form, which is what we proceed to do:
Let
c
k
=
1
2
(a
k
ib
k
)
and
c
k
= c
k
With these denitions, we have the following relations:
a
k
= c
k
+ c
k
and
b
k
= i(c
k
c
k
)
So
c
k
=
1
2
(a
k
ib
k
) =
1
2
_
2
0
f(x)(cos kx i sin kx) dx
Using Eulers theorem, we have:
c
k
=
1
2
_
2
0
f(x)e
ikx
dx
Finally, notice that
c
k
e
ikx
+ c
k
e
ikx
= c
k
(cos kx + i sin kx) + c
k
(cos kx i sin kx) = (c
k
+ c
k
) cos kx + i(c
k
c
k
) sin kx
This gives:
c
k
e
ikx
+ c
k
e
ikx
= a
k
cos kx + b
k
sin kx
Therefore the nth degree Fourier series:
s
n
(f(x)) =
a
0
2
+
n

k=1
(a
k
cos kx + b
k
sin kx)
can be rewritten as:
s
n
(f(x)) =
n

k=n
c
k
e
ikx
Fourier series have many applications. One of the most important is in the area of signal processing. The
Fourier series of a noisy function can be used to reduce this noise. Noise in this case refers to the terms of
s
n
(f(x)) where |k| is large. We can eliminate this noise by setting the corresponding c
k
to 0.
2
3. Discrete Fourier Transform
In applications, we often do not know f(x) explicitly, rather we have to approximate f(x) by sampling. This
will lead to an approximation of s
n
(f(x)).
First, we partition the interval [0, 2] into N equal parts:
x
0
= 0, x
1
=
2
N
, . . . , x
N
= 2
In particular,
x
j
=
2j
N
For each x
j
we measure the output y
j
and set
y
j
= f(x
j
)
Since we do not know f(x), we can not compute c
k
=
1
2
_
2
0
f(x)e
ikx
dx. Rather we will approximate this
integral using the left endpoint Riemann sum given the partition above:
c
k

N1

j=0
f(x
j
)e
ikxj
1
N
Note that
e
ikxj
= e

2ikj
N
=
jk
N
where
N
is the primitive Nth root of unity dened in section 1. Therefore
N1

j=0
f(x
j
)e
ikxj
1
N
=
1
N
N1

j=0
y
j

jk
N
We dene
d
k
=
N1

j=0
y
j

jk
N
The sequence {d
0
, d
1
, . . . , d
N1
} is called the discrete Fourier transform or DFT of {y
0
, y
1
, . . . , y
N1
}
We will now show how to compute d
k
using matrix multiplication. Let F
N
be the N N matrix whose k, j
entry is
(j1)(k1)
N
so that
F
N
=
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1 1 1 1 1
1
N

2
N

3
N

N1
N
1
2
N

4
N

6
N

2(N1)
N
1
3
N

6
N

9
N

3(N1)
N
.
.
.
.
.
.
.
.
.
.
.
.
1
N1
N

2(N1)
N

3(N1)
N

(N1)(N1)
N
_
_
_
_
_
_
_
_
_
_
_
_
_
_
Note that F
N
is a symmetric matrix (ie. F
T
N
= F
N
).
If we dene y and d by:
y =
_
_
_
_
_
_
_
_
y
0
y
1
.
.
.
y
N1
_
_
_
_
_
_
_
_
and d =
_
_
_
_
_
_
_
_
d
0
d
1
.
.
.
d
N1
_
_
_
_
_
_
_
_
Then a direct calculation shows that
F
N
y = d
3
4. Fast Fourier Transform
Using the matrix calculation above to compute the discrete Fourier transform of a sequence {y
0
, y
1
, . . . , y
N1
}
requires approximately 8N
2
operations. There are many algorithms which can be used to speed this processes
up, they are collectively known as the fast Fourier transform or FFT. The best known of these is the Cooley-
Tukey algorithm. This algorithm was developed by Cooley and Tukey in the 1960s, though it was later found
to be originally developed by Gauss in the early 1800s. This algorithm works when N is even and relies upon
reordering the columns of F
N
.
To get started, we dene two matrices as follows: Let {e
1
, e
2
, . . . , e
N
} be the standard basis for R
n
. Then
P
N
is dened to be:
P
N
= (e
1
e
3
e
5
e
N1
e
2
e
4
e
N
)
Notice that P
N
is a N N permutation matrix, in particular P
1
N
= P
T
N
. We also dene D
N/2
to be the
N/2 N/2 diagonal matrix whose j, j entry is
j1
N
.
Our rst goal is to rearrange the columns of F
N
so that all of the odd numbered columns occur before all of
the even columns. This can be done by multiplying F
N
on the right by P
N
. This has the eect of arranging the
columns so that the rst N/2 columns contain only even powers of
N
. We then use the relations
2
N
=
N/2
and
k(N/2+1)
N/2
=
k
N/2
to reduce the entries to powers of
N/2
. These columns are now seen to consist of two
copies of F
N/2
. For the remaining columns, we can do a similar reduction, with the added complication that we
have a copy of D
N/2
F
N/2
and of D
N/2
F
N/2
. Once we do this, we have the following block decomposition of
F
N
:
F
N
P
N
=
_
_
F
N/2
D
N/2
F
N/2
F
N/2
D
N/2
F
N/2
_
_
So,
F
N
y = (F
N
P
N
)(P
T
N
y)
If we write
P
T
N
y =
_
_
w
1
w
2
_
_
where w
1
is the rst half of of P
T
N
y and w
2
is the second half of this vector and use the properties of block
multiplication (see section 1.5 of Leon):
F
N
y =
_
_
F
N/2
D
N/2
F
N/2
F
N/2
D
N/2
F
N/2
_
_
_
_
w
1
w
2
_
_
=
_
_
F
N/2
w
1
+ D
N/2
F
N/2
w
2
F
N/2
w
1
D
N/2
F
N/2
w
2
_
_
Therefore, to compute F
N
y we only need to compute F
N/2
w
1
and D
N/2
F
N/2
w
2
. This process can be used
recursively, in stages: one stage for each power of 2 which divides N. This algorithm is most ecient when N
is a power of 2. In this case the FFT takes approximately 5N log
2
N operations (as opposed to 8N
2
operations
for the DFT). For example, if N = 1024 then the dft takes 2
23
operations whereas the FFT will take 2
11
25
operations. This is speedup by a factor of 2
12
/25.
4
Example:
Let N = 6 and y = (1, 2, 3, 4, 5, 6)
T
.
Then
F
6
=
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1 1 1 1 1 1
1
6

2
6

3
6

4
6

5
6
1
2
6

4
6

6
6

8
6

10
6
1
3
6

6
6

9
6

12
6

15
6
1
4
6

8
6

12
6

16
6

20
6
1
5
6

10
6

15
6

20
6

25
6
_
_
_
_
_
_
_
_
_
_
_
_
_
_
It is advantageous to reduce the various powers of
6
to smaller powers of
6
and powers of lower order
primitive roots of unity (also note that
3
6
= 1,
5
6
=
3
and
6
=
2
3
):
F
6
=
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1 1 1 1 1 1
1
2
3

3
1
2
3

3
1
3

2
3
1
3

2
3
1 1 1 1 1 1
1
2
3

3
1
2
3

3
1
3

2
3
1
3

2
3
_
_
_
_
_
_
_
_
_
_
_
_
_
_
P
6
=
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1 0 0 0 0 0
0 0 0 1 0 0
0 1 0 0 0 0
0 0 0 0 1 0
0 0 1 0 0 0
0 0 0 0 0 1
_
_
_
_
_
_
_
_
_
_
_
_
_
_
F
3
=
_
_
_
_
_
1 1 1
1
3

2
3
1
2
3

4
3
_
_
_
_
_
=
_
_
_
_
_
1 1 1
1
3

2
3
1
2
3

3
_
_
_
_
_
D
3
=
_
_
_
_
_
1 0 0
0
6
0
0 0
2
6
_
_
_
_
_
=
_
_
_
_
_
1 0 0
0
2
3
0
0 0
3
_
_
_
_
_
and
D
3
F
3
=
_
_
_
_
_
1 1 1

2
3
1
3

3
1
2
3
_
_
_
_
_
5
Now compute F
6
P
6
:
F
6
P
6
=
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1 1 1 1 1 1
1
3

2
3

2
3
1
3
1
2
3

3

3
1
2
3
1 1 1 1 1 1
1
3

2
3

2
3
1
3
1
2
3

3

3
1
2
3
_
_
_
_
_
_
_
_
_
_
_
_
_
_
=
_
_
F
3
D
3
F
3
F
3
D
3
F
3
_
_
Therefore
d = F
6
y = F
6
P
6
P
T
6
y =
_
_
F
3
D
3
F
3
F
3
D
3
F
3
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1
3
5
2
4
6
_
_
_
_
_
_
_
_
_
_
_
_
_
_
=
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
F
3
_
_
_
_
_
1
3
5
_
_
_
_
_
+ D
3
F
3
_
_
_
_
_
2
4
6
_
_
_
_
_
F
3
_
_
_
_
_
1
3
5
_
_
_
_
_
D
3
F
3
_
_
_
_
_
2
4
6
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
Finally,
F
3
_
_
_
_
_
1
3
5
_
_
_
_
_
=
_
_
_
_
_
1 1 1
1
3

2
3
1
2
3

3
_
_
_
_
_
_
_
_
_
_
1
3
5
_
_
_
_
_
=
_
_
_
_
_
9
1 + 3
3
+ 5
2
3
1 + 3
2
3
+ 5
3
_
_
_
_
_
and
D
3
F
3
_
_
_
_
_
2
4
6
_
_
_
_
_
=
_
_
_
_
_
1 1 1

2
3
1
3

3
1
2
3
_
_
_
_
_
_
_
_
_
_
2
4
6
_
_
_
_
_
=
_
_
_
_
_
12
2
2
3
4 6
3
2
3
+ 4 + 6
2
3
_
_
_
_
_
So
d = F
6
y =
1
6
_
_
_
_
_
_
_
_
_
_
_
_
_
_
9 + 12
3 3
3
+ 3
2
3
5 + 7
3
+ 9
2
3
9 12
5 + 9
3
+ 7
2
3
3 + 3
3
3
2
3
_
_
_
_
_
_
_
_
_
_
_
_
_
_
=
1
6
_
_
_
_
_
_
_
_
_
_
_
_
_
_
21
3.0000 + 5.1962i
3.0000 + 1.7321i
3.0000
3.0000 1.7321i
3.0000 5.1962i
_
_
_
_
_
_
_
_
_
_
_
_
_
_
6