Signal Processing MIT PDF

MIT OpenCourseWare
http://ocw.mit.edu
2.161 Signal Processing: Continuous and Discrete

Fall 2008
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Massachusetts Institute of Technology
Department of Mechanical Engineering
2.161 Signal Processing - Continuous and Discrete

Fall Term 2008
Lecture 11
Reading:
Class handout: The Dirac Delta and Unit-Step Functions
1 Introduction to Signal Processing

In this class we will primarily deal with processing time-based functions, but the methods
will also be applicable to spatial functions, for example image processing. We will deal with
(a) Signal processing of continuous waveforms f (t), using continuous LTI systems (lters).
a L T I d y n a m i c a l s y s t e m
in p u t o u tp u t
f(t) C o n tin u o u s S ig n a l y (t)
P ro c e s s o r
and
(b) Discrete-time (digital) signal processing of data sequences {fn } that might be samples of
real continuous experimental data, such as recorded through an analog-digital converter
(ADC), or implicitly discrete in nature.
a L T I d i s c r e t e a l g o r i t h m
i n p u t s e q u e n c e o u tp u t s e q u e n c e
{fn } D is c r e te S ig n a l {y n }
P ro c e s s o r
Some typical applications that we look at will include
(a) Data analysis, for example estimation of spectral characteristics, delay estimation in
echolocation systems, extraction of signal statistics.
(b) Signal enhancement. Suppose a waveform has been contaminated by additive noise,
for example 60Hz interference from the ac supply in the laboratory.
1 c D.Rowell 2008
copyright
11
in p u t o u tp u t
+
f(t) F ilte r y (t)
+
n ( t )
a d d i t i v e n o i s e
The task is to design a lter that will minimize the eect of the interference while not
destroying information from the experiment.
(c) Signal detection. Given a noisy experimental record, ask the question whether a known
signal is present in the data.
1.1 Processing methods

(a) Passive Continuous Filters: We will investigate signal processing using passive con
tinuous LTI (Linear Time-Invariant) dynamical systems. These will typically be elec
trical R-L-C systems, for example
R L
V in (t) C v o (t) V i n (t) C C v (t)

1 2 o
or even an electro mechanical system using rotational elements:
Is ( t ) a c tu a to r filte r s e n s o r
+ +
v o (t)
- -
J 1 J 2
K 1
K 2
B 1 B 2
(b) Active Continuous Filters: Modern continuous lters are implemented using oper
ational ampliers. We will investigate simple op-amp designs.
C 1
R 3
R 1 C 2
-
+
+
V in R 2
+
V o u t
12
(c) Digital Signal Processors: Here a digital system (a computer or DSP chip) is used
to process a data stream.
D ig ita l S ig n a l
P ro c e s s o r
re c o n s tru c to r
s a m p le r
f
( t )
P ro c e s s e s d is c r e te s a m p le s y ( t )
o f th e in p u t f n = f(n T ) to
p ro d u c e a d is c r e te o u tp u t
s e q u e n c e y n .
(i) The sampler (A/D converter) records the signal value at discrete times to produce
a sequence of samples {fn } where fn = f (nT ) (T is the sampling interval.

f(t)
t
0 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T
(ii) At each interval, the output sample yn is computed, based on the history of the
input and output, for example
yn = (fn + fn1 + fn2 )

3
3-point moving average lter, and
yn = 0.8yn1 + 0.2fn
is a simple recursive rst-order low-pass digital lter. Notice that they are algo
rithms.
(iii) The reconstructor takes each output sample and creates a continuous waveform.
In real-time signal processing the system operates in an innite loop:
f(t)
s a m p le f( t)
t
f
n
c o m p u te y
y (t) s ta ir c a s e o r n
z e ro -o rd e r
w a v e fo rm
re c o n s tru c t y (t)
0 t
0 tim e
13
2 Properties of LTI Continuous Filters

in p u t o u tp u t
u (t) L T I C o n tin u o u s F ilte r y (t)
A LTI lter is dynamical SISO (single-input single-output) linear system, governed by an

ODE with constant coecients. From elementary linear system theory, some fundamental
properties of LTI systems are:
(a) The Principle of Superposition This is the fundamental property of linear systems.
For a system at rest at time t = 0, if the response to input f1 (t) is y1 (t), and the response
to f2 (t) is y2 (t), then the response to a linear combination of f1 (t) and f2 (t), that is
f3 (t) = af1 (t) + bf2 (t) (a and b constants) is
y3 (t) = ay1 (t) + by2 (t).
(b) The Dierentiation Property If the response to input f (t) is y(t), then the re
sponse to the derivative of f (t), that is f1 (t) = df /dt is
dy
y1 (t) = .
dt
(c) The Integration Property If the response t to input f (t) is y(t), then the response
to the integral of f (t), that is f1 (t) = f (t)dt is
t
y1 (t) = y(t)dt.

(d) Causality A causal system is non-anticipatory, that is it does not respond to an input
before it occurs. Physical LTI systems are causal.
3 The Dirac Delta Function

The Dirac delta function is a non-physical, singularity function with the following denition

0 for t = 0
(t) =
undened at t = 0
but with the requirement that

(t)dt = 1,

that is, the function has unit area. Despite its name, the delta function is not truly a
function. Rigorous treatment of the Dirac delta requires measure theory or the theory of
distributions.
14
d T (t) d (t)
1 /T
1
1 /T
2
1 /T
3
1 /T
4
t t
0 T T T T 0
1 2 3 4
a ) U n it p u ls e s o f d iffe r e n t e x te n ts b ) T h e im p u ls e fu n c tio n
The gure below shows a unit pulse function T (t), that is a brief rectangular pulse
function of extent T , dened to have a constant amplitude 1/T over its extent, so that the
area T 1/T under the pulse is unity:

0 for t 0
T (t) = 1/T 0<tT

0
for t > T .
The Dirac delta function (also known as the impulse function) can be dened as the limiting
form of the unit pulse T (t) as the duration T approaches zero. As the extent T of T (t)
decreases, the amplitude of the pulse increases to maintain the requirement of unit area
under the function, and
(t) = lim T (t).
T 0
The impulse is therefore dened to exist only at time t = 0, and although its value is strictly
undened at that time, it must tend toward innity so as to maintain the property of unit
area in the limit.
4 Properties of the Delta Function

4.0.1 Time Shift
An impulse occurring at time t = a is (t a).
4.0.2 The strength of an impulse

Because the amplitude of an impulse is innite, it does not make sense to describe a scaled
impulse by its amplitude. Instead, the strength of a scaled impulse K(t) is dened by its
area K.
4.0.3 The Sifting Property of the Impulse

When an impulse appears in a product within an integrand, it has the property of sifting
out the value of the integrand at the time of its occurrence:

f (t)(t a)dt = f (a)

15
This is easily seen by noting that (t a) is zero except at t = a, and for its innitesimal
duration f (t) may be considered a constant and taken outside the integral, so that

f (t)(t a)dt = f (a) (t a)dt = f (a)

from the unit area property.
4.0.4 Scaling
A helpful identity is the scaling property:

du 1
(t)dt = (u) =
|| ||
and so
1
(t) = (t).
||
4.0.5 Laplace Transform

L {(t)} = (t)est dt = 1
0
by the sifting property.
4.0.6 Fourier Transform

F {(t)} = (t)ejt dt = 1

by the sifting property.
5 Practical Applications of the Dirac Delta Function

The most important application of t in linear system theory is directly related to its
Laplace transform property, L {(t)} = 1. Consider a SISO LTI system with transfer
function H(s), with input u(t) and output y(t), so that in the Laplace domain
Y (s) = H(s)U (s).
If the input is u(t) = (t), so that U (s) = 1, then Y (s) = H(s).1, and through the
inverse Laplace transform
y(t) = h(t) = L1 {H(s)} .
where h(t) is dened as the systems impulse response. The impulse response com
pletely characterizes the system, in the sense that it allows computation of the transfer
function (and hence the dierential equation).
16
The impulse response h(t) is used in the convolution integral.
In signal processing the delta function is used to create a Dirac comb (also known as
an impulse train, or Shah function):

T (t) = (t nT )
n=
is used in sampling theory. A continuous waveform f (t) is sampled by multiplication

by the Dirac comb

f (t) = f (t)T (t) = f (t nT )(t nT ),
n=
where f (t) is the sampled waveform, producing a train of weighted impulses.
17
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 21
Reading:
Class handout: Convolution
Class handout: Sinusoidal Frequency Response
1 Continuous LTI System Time-Domain Response

A continuous linear lter is a LTI dynamical system (described by an ODE with constant
coecients). We are interested in the input-output relationships and seek a method of
determining the response y(t) to a given input u(t).
u (t)
u (t) L T I F ilte r y (t) = ?

in p u t o u tp u t
t
s y s te m is a t r e s t a t t= 0
The relationship is developed as follows (see the handout for a detailed explanation)
The input u(t) is approximated as a zero-order (staircase) waveform uT (t) with intervals
T.

uT (t) = u(nT ) for nT t < (n + 1)T.
The approximation uT (t) is written as a superposition of non-overlapping pulses

uT (t) = pn (t)
n=
1 c D.Rowell 2008
copyright
21
where
u(nT ) nT t < (n + 1)T
pn (t) =
0 otherwise
For example, p3 (t) is shown cross-hatched in the gure above.
Each component pulse pn (t) is written in terms of a delayed unit pulse T (t), of width
T and amplitude 1/T that is:
pn (t) = u(nT )T (t nT )T
so that

uT (t) = u(nT )T (t nT )T.
n=
Assume that the system response to an input T (t) is a known function, and is desig
nated hT (t) as shown below. If the system is linear and time-invariant, the response
to a delayed unit pulse, occurring at time nT is simply a delayed version of the pulse
response:
yn (t) = hT (t nT )
d T (t-n T ) y n
( t )
1 / T
d T (t-n T ) s y s te m y n ( t )
0 t 0
0 n T ( n + 1 ) T t
0 n T (n + 1 )T
The principle of superposition allows the total system response to uT (t) to be written
as the sum of the responses to all of the component weighted pulses:

yT (t) = u(nT )hT (t nT )T
n=

For causal systems the pulse response hT (t) is zero for time t < 0, and future com
ponents of the input do not contribute to the sum, so that the upper limit of the
summation may be rewritten:

N
yT (t) = u(nT )hT (t nT )T for N T t < (N + 1)T.
n=
22
We now let the pulse width T become very small, and write nT = , T = d , and note
that limT 0 T (t) = (t). As T 0 the summation becomes an integral and

N
y(t) = lim u(nT )hT (t nT )T
T 0
n=
t
= u( )h(t )d (1)

where h(t) is dened to be the system impulse response,
h(t) = lim hT (t).

T 0
Equation (??) is an important integral in the study of linear systems and is known as the
convolution or superposition integral. It states that the system is entirely characterized by its
response to an impulse function (t), in the sense that the forced response to any arbitrary
input u(t) may be computed from knowledge of the impulse response alone. The convolution
operation is often written using the symbol :
t
y(t) = u(t) h(t) = u( )h(t )d. (2)

Equation (??) is in the form of a linear operator, in that it transforms, or maps, an input
function to an output function through a linear operation.
c o n v o lu tio n
u (t)
L T I F ilte r
u (t) y ( t ) = u ( t ) OX h ( t )
in p u t h (t) o u tp u t
t
The form of the integral in Eq. (??) is dicult to interpret because it contains the term
h(t ) in which the variable of integration has been negated. The steps implicitly involved
in computing the convolution integral may be demonstrated graphically below. The impulse
response h( ) is reected about the origin to create h( ), and then shifted to the right
by t to form h(t ). The product u(t)h(t ) is then evaluated and integrated to nd
the response. This graphical representation is useful for dening the limits necessary in the
integration. For example, since for a physical system the impulse response h(t) is zero for all
t < 0, the reected and shifted impulse response h(t ) will be zero for all time > t. The
upper limit in the integral is then at most t. If in addition the input u(t) is time limited,
that is u(t) 0 for t < t1 and t > t2 , the limits are:
t

u( )h(t )d for t < t2
yf (t) = t2
t1 (3)

u( )h(t )d for t t2
t1
23
See the class handout for further details and examples.
24
2 Sinusoidal Response of LTI Continuous Systems

Of particular interest is the response of an LTI continuous system to sinusoidal inputs of the
form u(t) = A sin(t + ), where A is the amplitude, is the angular frequency (rad/s),
and is a phase angle (rad). (We note that we can also write u(t) = A sin(2F t + ), where
F is the frequency in Hz.)
We begin by noting that a sinusoid may be expressed in terms of complex exponentials
through the Euler formulas:
1 jt
sin(t) = e ejt
2j
1 jt
cos(t) = e + ejt
2
and rst nding the steady-state solution to inputs of the form u(t) = ejt . Let the LTI
system be described by an ODE of the form
dn y dn1 y dy dm u dm1 u du
an n
+ an1 n1
+ + a 1 + a 0 y = b m n
+ b m1 m1
+ + b1 + b0 u.
dt dt dt dt dt dt
The steady-state response of the system (after all initial condition transients have decayed)
may be found using the method of undetermined coecients, in which a form of the solution
is assumed and solved for a set of coecients. In particular, if the input is u(t) = ejt ,
assume that
y(t) = Bejt .
Substitution into the dierential equation gives

an (j)n + an1 (j)n1 + + a1 (j) + a0 Bejt

= bm (j)m + bn1 (j)m1 + + b1 (j) + b0 ejt
and solving for B
an (j)n + an1 (j)n1 + + a1 (j) + a0

B=
bm (j)m + bn1 (j)m1 + + b1 (j) + b0
so that
y(t) = H(j)ejt
where
N (j) an (j)n + an1 (j)n1 + + a1 (j) + a0
H(j) = =
D(j) bm (j)m + bn1 (j)m1 + + b1 (j) + b0
H(j) is dened to be the frequency response function, and N (j) and D(j) are the
numerator and denominator polynomials respectively. We note the following:
The output y(t) is simply a (multiplicatively) weighted version of the input.
H(j) is a property of the system. It is dened entirely by the describing dierential

equation.
25
H(j) is, in general, complex. Even powers of n and m in N (s) and D(s) will generate
real terms in the polynomials, while odd powers will generate imaginary terms.
|N (j)|
|H(j)| =
|D(j)|
H(j) = N (j) D(j)
H(j) = H(j), where H(j) is the complex conjugate.

The response to the real sinusoid
A j(t+)
u(t) = A sin (t + ) = e ej(t+)

2j
may be found from the principle of superposition by summing the response to each compo
nent:
A
y(t) = H(j)ej(t+) H(j)ej(t+)

2j
A
= H(j)ej(t+) H(j)ej(t+)
2j
Combining the real and imaginary parts gives the result
y(t) = A |H(j)| sin (t + + H(j))
where |H(j)| is the magnitude of the frequency response function, and H(j) is the phase
response.
The response to a real sinusoid is therefore a sinusoid of the same frequency as the
input.
The amplitude of the response at an input frequency of has been modied by a factor
|H(j)|. If |H(j)| > 1 the input has been amplied by the system, if |H(j)| < 1,
the signal has been attenuated.
The system has imposed a frequency dependent phase shift H(j) on the response.
Example 1
A rst-order passive RC lter with the following circuit diagram
+
V in (t) C v (t)
o
-
26
is described by the dierential equation

dvo
RC
+ vo = Vin (t)
dt
Find the frequency response function.
By inspection
1
H(j) =
jRC + 1
and
|1| 1
|H(j)| = =
|1 + jRC| (RC)2 + 1
H(j) = (1) (1 + jRC) = 0 tan1 (RC)
Clearly, as 0, |H(j)| 1, and H(j) 0 rad. As , |H(j)| 0,
and H(j) /2 rad (-90 ).
This is a low-pass lter, in that it passes low frequency sinusoids while attenuating
high frequencies.
Example 2
A new rst-order passive RC lter is formed by exchanging the resistor and
capacitor in the previous example:
C
+
V in (t) R v (t)
o
-
and is now described by the dierential equation

dvo dVin
RC
+ vo = RC
dt dt
Find the frequency response function.
By inspection
jRC
H(j) =
jRC + 1
and
|jRC| RC
|H(j)| = =
|1 + jRC| (RC)2 + 1

H(j) = (jRC) (1 + jRC) = tan1 (RC)
2
27
Clearly, as 0, |H(j)| 0, and H(j) /2 rad (90 ). As ,

|H(j)| 1, and H(j) 0 rad (0 ).
This is a high-pass lter, in that it attenuates low frequency sinusoids while
passing high frequencies.
28
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 31
Reading:
Class handout: Frequency Domain Methods
1 The Fourier Series and Transform

In Lecture 2 we looked at the response of LTI continuous systems to sinusoidal inputs of the
form
u(t) = A sin(t + )
and saw that the system was characterized by the frequency response function H(j).
In signal processing work, linear lters are usually specied by a desired frequency re
sponse function. (We will see that often the magnitude function |H(j)| alone is used to
specify a lter). The following gure shows the four basic forms of ideal linear lters:
|H ( jW ) | | H ( j W ) |
l o w - p a s s f i l t e r
h ig h - p a s s filte r
1
1
W W
- W c 0 W c - W c
0 W c
|H ( jW ) | |H ( jW ) |
b a n d - p a s s filte r b a n d - s to p filte r
1
1
W W
- W - W 0 W W - W c 0 W c
c 2 c 1 c 1 c 2
In this lecture we generalize the response of LTI systems to non-sinusoidal inputs. We do

this using Fourier methods.
1 c D.Rowell 2008
copyright
31
2 Periodic Input Functions - The Fourier Series
In general, a periodic function is a function that satises the relationship:
x(t) = x(t + T )
for all t, or x(t) = x(t + nT ) for n = 1, 2, 3, . . ., where T is dened as the period. Some
examples of periodic functions is shown below.
x 1 (t) x (t) x 3 (t) T

T 2 T
t t t
0 0
The fundamental angular frequency 0 (in radians/second) of a periodic waveform is dened

directly from the period
2
0 = ,
T
and the fundamental frequency F0 (in Hz) is
1
F0 =
T
so that 0 = 2F0 .
Any periodic function with period T is also be periodic at intervals of nT for any
positive integer n. Similarly any waveform with a period of T /n is periodic at intervals
of T seconds.
Two waveforms whose periods, or frequencies, are related by a simple integer ratio are
said to be harmonically related.
If two harmonically related functions are summed together to produce a new function
g(t) = x1 (t) + x2 (t), then g(t) will be periodic with a period dened by the longest
period of the two components. In general, when harmonically related waveforms are
added together the resulting function is also periodic with a repetition period equal to
the fundamental period.
32
Example 1
A family of waveforms gN (t) (N = 1, 2 . . . 5) is formed by adding together the
rst N of up to ve component functions, that is

N
gN (t) = xn (t) 1<N 5
n=1
where
x1 (t) = 1
x2 (t) = sin(2t)
1
x3 (t) = sin(6t)
3
1
x4 (t) = sin(10t)
5
1
x5 (t) = sin(14t).
7
The rst term is a constant, and the four sinusoidal components are harmonically
related, with a fundamental frequency of 0 = 2 rad/s and a fundamental period
of T = 2/0 = 1 second. (The constant term may be considered to be periodic
with any arbitrary period, but is commonly considered to have a frequency of
zero rad/s.) The gure below shows the evolution of the function that is formed
as more of the individual terms are included into the summation. Notice that
in all cases the period of the resulting gN (t) remains constant and equal to the
period of the fundamental component (1 second). In this particular case, it can
be seen that the sum is tending toward a square wave as more terms are included.
g (t) (n = 1 ...5 )
2 .0
N
g 5(t)
S y n tn th e s iz e d fu n c tio n
g 2 (t)
1 .5
g 1(t)
1 .0
0 .5
0 .0 t
0 .0 0 .5 1 .0 1 .5 2 . 0
T i m e ( s e c )
33
The Fourier series representation of an arbitrary periodic waveform x(t) (subject to some
general conditions described later) is as an innite sum of harmonically related sinusoidal
components, commonly written in the following three equivalent forms
1
x(t) = a0 + (an cos(n0 t) + bn sin(n0 t)) (1)
2 n=1
1
= a0 + An sin(n0 t + n ) (2)
2 n=1

+
= Xn ejn0 t . (3)
n=
In each representation knowledge of the fundamental frequency 0 , and the sets of Fourier
coecients {an } and {bn } (n = 0 . . . ), or {An } and {n }) (n = 0 . . . ), or {Xn } (n =
. . . ) is sucient to completely dene the waveform x(t).
These representations are related by

An = a2n + b2n
n = tan1 (an /bn ).
and
Xn = 1/2(an jbn )
Xn = 1/2(an + jbn )
See the class handout for details.

The spectrum of a periodic waveform is the set of all of Fourier coecients in any of
the representations, for example {An } and {n }, expressed as a function of frequency. Be
cause the harmonic components exist at discrete frequencies, periodic functions are said to
exhibit line (or discrete) spectra, and it is common to express the spectrum graphically with
frequency as the independent axis, and with the Fourier coecients plotted as lines at
intervals of 0 . The rst two forms of the Fourier series, based upon Eqs. (??) and (??),
generate one-sided spectra because they are dened from positive values of n only, whereas
the complex form dened by Eq. (??) generates a two-sided spectrum because its summa
tion requires positive and negative values of n. The gure below shows a complex spectrum
(Eq. ??).
34
{X n }
2 .0
1 .0
n
-5 -4 -3 -2 -1 0 1 2 3 4 5
{X n }
2 .0
1 .0
n
-5 -4 -3 -2 -1 0 1 2 3 4 5
-1 .0
-2 .0
2.1 Computation of the Fourier Coecients

Section (2.1) of the class handout derives the nite (truncated) Fourier series as a least-
squares approximation to the periodic function x(t). The results show that for the complex
representation
1 t0 +T
Xn = xp (t)ejn0 t dt,
T t0
and for the real representation
t0 +T
2
an = x(t) cos(n0 t)dt
T t0
t0 +T
2
bn = x(t) sin(n0 t)dt.
T t0
The results are summarized in the following table:

Sinusoidal formulation Exponential formulation
1
+
Synthesis: x(t) = a0 + (an cos(n0 t) + bn sin(n0 t)) x(t) = Xn ejn0 t
2 n=1 n=

2 t1 +T 1 t1 +T
Analysis: an = x(t) cos(n0 t)dt Xn = x(t)ejn0 t dt
T t1 T
t1 +T t1
2
bn = x(t) sin(n0 t)dt
T t1
35
2.2 Properties of the Fourier Series

The following are some of the important properties of the Fourier series:
(1) Existence of the Fourier Series For the series to exist, the Fourier analysis integral
must converge. A set of three sucient conditions, known as the Dirichelet conditions,
guarantee the existence of a Fourier series for a given periodic waveform x(t). They
are
The function x(t) must be absolutely integrable over any period, that is
t0 +T
|x(t)| dt <
t0
for any t0 .
There must be at most a nite number of maxima and minima in the function
x(t) within any period.
There must be at most a nite number of discontinuities in the function x(t)
within any period, and all such discontinuities must be nite in magnitude.
These requirements are satised by almost all waveforms found in engineering practice.
The Dirichelet conditions are a sucient set of conditions to guarantee the existence
of a Fourier series representation. They are not necessary conditions, and there are
some functions that have a Fourier series representation without satisfying all three
conditions.
(2) Linearity of the Fourier Series Representation The Fourier analysis and synthe
sis operations are linear. Consider two periodic functions g(t) and h(t) with identical
periods T , and their complex Fourier coecients

1 T
Gn = g(t)ejn0 t dt
T 0

1 T
Hn = h(t)ejn0 t dt
T 0
and a third function dened as a weighted sum of g(t) and h(t)
x(t) = ag(t) + bh(t)
where a and b are constants. The linearity property, which may be shown by direct
substitution into the integral, states that the Fourier coecients of x(t) are
Xn = aGn + bHn ,
that is the Fourier series of a weighted sum of two time-domain functions is the weighted
sum of the individual series.
36
(3) Even and Odd Functions If x(t) exhibits symmetry about the t = 0 axis the Fourier
series representation may be simplied. If x(t) is an even function of time, that is
x(t) = x(t), the complex Fourier series has coecients Xn that are purely real, with
the result that the real series contains only cosine terms, so that Eq. (??) simplies to
1
x(t) = a0 + an cos(n0 t).
2 n=1
Similarly if x(t) is an odd function of time, that is x(t) = x(t), the coecients Xn
are imaginary, and the one-sided series consists of only sine terms:

x(t) = bn sin(n0 t).
n=1
Notice that an odd function requires that x(t) have a zero average value.
(4) The Fourier Series of a Time Shifted Function If the periodic function x(t) has
a Fourier series with complex coecients Xn , the series representing a time-shifted
version g(t) = x(t + ) has coecients ejn0 Xn . If
T
1
Xn = x(t)ejn0 t dt
T 0
then
T
1
Gn = f (t + )ejn0 t dt.
T 0
Changing the variable of integration = t + gives

1 +T
Gn = f ()ejn0 ( ) d
T
+T
jn0 1
= e f ()ejn0 t d
T
= ejn0 Xn .
If the nth spectral component is written in terms of its magnitude and phase
xn (t) = An sin(n0 t + n )
then
xn (t + ) = An sin (n0 (t + ) + n )
= An sin (n0 t + n + n0 ) .
The additional phase shift n0 , caused by the time shift , is directly proportional
to the frequency of the component n0 .
37
(5) Interpretation of the Zero Frequency Term The coecients X0 in the complex
series and a0 in the real series are somewhat dierent from all of the other terms for
they correspond to a harmonic component with zero frequency. The complex analysis
equation shows that
1 t1 +T
X0 = x(t)dt
T t1
and the real analysis equation gives

1 1 t1 +T
a0 = x(t)dt
2 T t1
which are both simply the average value of the function over one complete period.
If a function x(t) is modied by adding a constant value to it, the only change in its
series representation is in the coecient of the zero-frequency term, either X0 or a0 .
2.3 The Response of Linear Systems to Periodic Inputs

Consider a linear single-input, single-output system with a frequency response function
H(j). Let the input u(t) be a periodic function with period T , and write it in terms
of a real Fourier series:
1
u(t) = a0 + An sin(n0 t + n )
2 n=1
The nth real harmonic input component, un (t) = An sin(n0 t + n ), generates an output
sinusoidal component yn (t) with a magnitude and a phase that is determined by the systems
frequency response function H(j):
yn (t) = |H(jn0 )| An sin(n0 t + n + H(jn0 )).
The principle of superposition states that the total output y(t) is the sum of all such com
ponent outputs, or

y(t) = yn (t)
n=0
1
= a0 H(j0) + An |H(jn0 )| sin (n0 t + n + H(jn0 )) ,
2 n=1
which is itself a Fourier series with the same fundamental and harmonic frequencies as the
input. The output y(t) is therefore also a periodic function with the same period T as
the input, but because the system frequency response function has modied the relative
magnitudes and the phases of the components, the waveform of the output y(t) diers in
form and appearance from the input u(t).
In the complex formulation the input waveform is decomposed into a set of complex
exponentials un (t) = Un ejn0 t . Each such component is modied by the system frequency
response so that the output component is
yn (t) = H(jn0 )Un ejn0 t
38
and the complete output Fourier series is

y(t) = yn (t) = H(jn0 )Un ejn0 t .
n= n=
y (t) p e r io d ic o u tp u t
u (t) p e r io d ic in p u t
y (t)
u (t) t
L T I F ilte r
t
T T
F o u r ie r a n a ly s is H ( jW ) F o u r ie r s y n th e s is

Sn =

U e
jn W o t m u ltip lic a tio n Sn = -
U n H ( jn W o ) e
jn W o t
- n
The system H(j) acts as a frequency-domain lter and modies the input wave
form u(t) by (1) selectively amplifying/attenuating the spectral components, and (2)
applying a frequency dependent phase shift.
See the class handout for examples.
3 Aperiodic Input Functions - The Fourier Transform

Many waveforms found in practice are not periodic and therefore cannot be analyzed directly
using Fourier series methods. A large class of system excitation functions can be character
ized as aperiodic, or transient, in nature. These functions are limited in time, they occur
only once, and decay to zero as time becomes large.
Consider a function x(t) of duration that exists only within a dened interval t1 < t
t1 + , and is identically zero outside of this interval. We begin by making a simple assump
tion; namely that in observing the transient phenomenon x(t) within any nite interval that
encompasses it, we have observed a fraction of a single period of a periodic function with a
very large period; much larger than the observation interval. Although we do not know what
the duration of this hypothetical period is, it is assumed that x(t) will repeat itself at some
time in the distant future, but in the meantime it is assumed that this periodic function
remains identically zero for the rest of its period outside the observation interval.
The analysis thus conjectures a new function xp (t), known as a periodic extension of x(t),
that repeats every T seconds (T > ), but at our discretion we can let T become very large.
39
x (t)
x p
(t)
x (t)
t
-2 T -T 0 D T 2 T 3 T
As observers of the function xp (t) we need not be concerned with its pseudo-periodicity
because we will never be given the opportunity to experience it outside the rst period, and
furthermore we can assume that if xp (t) is the input to a linear system, T is so large that the
system response decays to zero before the arrival of the second period. Therefore we assume
that the response of the system to x(t) and xp (t) is identical within our chosen observation
interval. The important dierence between the two functions is that xp (t) is periodic, and
therefore has a Fourier series description.
The development of Fourier analysis methods for transient phenomena is based on the
limiting behavior of the Fourier series describing xp (t) as the period T approaches innity.
The derivation proceeds in the following steps (see the class handout for details).
(1) The waveform xp (t) is described by a Fourier series with lines spaced at intervals
2
0 =
T
and coecients t0 +T
1
Xn = xp (t)ejn0 t dt
T t0
(2) From the synthesis equation (with t0 = T /2)

xp (t) = Xn ejn0 t
n=

0 T /2
jn0 t
= xp (t)e dt ejn0 t
n=
2 T /2
The gure below shows how the line spectrum varies as the period T changes. Note that
the shape of the envelope dening the spectrum is unaltered, but the the magnitude
and the line spacing changes.
310
x (t) T X n
T = 4
1 .0
w 0 = p /2 ra d /s e c
t n
0 1 2 3 4 0
x (t) T X n
T = 2
1 .0
w 0 = p ra d /s e c
t n
0 1 2 3 4
0
x (t) T X n
T = 1
1 .0
w 0 = 2 p ra d /s e c
t n
0 1 2 3 4 0
(c) The period T is now allowed to become arbitrarily large, with the result that the fun
damental frequency 0 becomes very small and we write 0 = . We dene x(t) as
the limiting case of xp (t) as T approaches innity, that is
x(t) = lim xp (t)

T

1 T /2
= lim xp (t)ejnt dt ejnt
T
n=
2 T /2

1 jt
= x(t)e dt ejt d (4)
2
where in the limit the summation has been replaced by an integral.
(d) If the function inside the braces is dened to be X(j), Eq. (??) may be expanded into
a pair of equations, known as the Fourier transform pair:

X(j) = x(t)ejt dt

1
x(t) = X(j)ejt d
2
which are the equations we seek.
(To be continued in Lecture 4)
311
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 41
Reading:

1 Review of Development of Fourier Transform:

We saw in Lecture 3 that the Fourier transform representation of aperiodic waveforms can be
expressed as the limiting behavior of the Fourier series as the period of a periodic extension
is allowed to become very large, giving the Fourier transform pair

X(j) = x(t)ejt dt (1)

1
x(t) = X(j)ejt d (2)
2
Equation (??) is known as the forward Fourier transform, and is analogous to the analysis
equation of the Fourier series representation. It expresses the time-domain function x(t)
as a function of frequency, but unlike the Fourier series representation it is a continuous
function of frequency. Whereas the Fourier series coecients have units of amplitude, for
example volts or Newtons, the function X(j) has units of amplitude density, that is the
total amplitude contained within a small increment of frequency is X(j)/2.
Equation (??) denes the inverse Fourier transform. It allows the computation of the
time-domain function from the frequency domain representation X(j), and is therefore
analogous to the Fourier series synthesis equation. Each of the two functions x(t) or X(j)
is a complete description of the function and the pair allows the transformation between the
domains.
We adopt the convention of using lower-case letters to designate time-domain functions,
and the same upper-case letter to designate the frequency-domain function. We also adopt
the nomenclature
F
x(t) X(j)
as denoting the bidirectional Fourier transform relationship between the time and frequency-
domain representations, and we also frequently write
X(j) = F{x(t)}
x(t) = F 1 {X(j)}
as denoting the operation of taking the forward F{}, and inverse F 1 {} Fourier transforms
respectively.
1 c D.Rowell 2008
copyright
41
1.1 Alternate Denitions
Although the denitions of Eqs. (??) and (??) ow directly from the Fourier series, denitions
for the Fourier transform vary from text to text and in dierent disciplines. The main
objection to the convention adopted here is the asymmetry introduced by the factor 1/2
that appears in the inverse transform. Some authors, usually in physics texts, dene the
so-called unitary Fourier transform pair as

1
X(j) = x(t)ejt dt
2

1
x(t) = X(j)ejt d
2
so as to distribute the constant symmetrically over the forward and inverse transforms.
Many engineering texts address the issue of the asymmetry by dening the transform
with respect to frequency F = 2 in Hz, instead of angular frequency in radians/s.
The eect, through the change in the variable in the inverse transform, is to redene the
transform pair as

X(F ) = x(t)ej2F t dt

x(t) = X(F )ej2F t dF

Some authors also adopt the notation of dropping the j from the frequency domain repre
sentation and write X() or X(F ) as above.
Even more confusing is the fact that some authors (particularly in physics) adopt a
denition that reverses the sign convention on the exponential terms in the Fourier integral,
that is they dene

X(j) = x(t)ejt dt

1
x(t) = X(j)ejt d
2
These various denitions of the transform pair mean that care must be taken to understand
the particular denition adopted by texts and software packages. Throughout this course
we will retain the denitions in Eqs. (??) and (??).
1.2 Fourier Transform Examples
Example 1
Find the Fourier transform of the pulse function

a |t| < T /2
x(t) =
0 otherwise.
42
x (t) X ( jW )
a
a T
F o u r ie r T r a n s fo r m
t W
-T /2 0 T /2 -3 0 -2 0 -1 0 0 1 0 2 0 3 0
T T T T T T
Solution: From the denition of the forward Fourier transform

X(j) = x(t)ejt dt (i)

T /2
= a ejt dt (ii)
T /2
T /2

j jt
= a e (iii)
T /2

ja jT /2
= e ejT /2 (iv)

sin(T /2)
= aT . (v)
T /2
The Fourier transform of the rectangular pulse is a real function, of the form
(sin x)/x centered around the j = 0 axis. Because the function is real, it
is sucient to plot a single graph showing only |X(j)|. Notice that while
X(j) is a generally decreasing function of it never becomes identically zero,
indicating that the rectangular pulse function contains frequency components at
all frequencies.
The function (sin x)/x = 0 when the argument x = n for any integer n (n = 0).
The main peak or lobe of the spectrum X(j) is therefore contained within
the frequency band dened by the rst two zero-crossings |T /2| < or || <
2/T . Thus as the pulse duration T is decreased, the spectral bandwidth of the
pulse increases, indicating that short duration pulses have a relatively larger high
frequency content.
43
X ( jW )
x (t) a T
a
t W
-T /2 0 T /2
X ( jW )
x (t)
a F o u r ie r T r a n s fo r m
T /4
a T /4
t W
-T /8 0 T /8
Example 2
Find the Fourier transform of the Dirac delta function (t).
Solution: When substituted into the forward Fourier transform

(j) = (t)ejt dt

= 1 (i)
by the sifting property. The spectrum of the delta function is therefore constant
over all frequencies. It is this property that makes the impulse a very useful test
input for linear systems.
Example 3
Find the Fourier transform of the causal real exponential function x(t) = us (t)eat
( for a > 0).
44
|X ( jW ) |
1 /a
x (t)
1 .0
W
-2 0 a -1 0 a 0 1 0 a 2 0 a
0 .5 e -a t |X ( jW ) |
1 .5
1 .0
a t
0 1 2 3 4 0 .5
W
-2 0 a -1 0 a 0 1 0 a 2 0 a
-0 .5
-1 .0
-1 .5
Solution: From the denition of the forward Fourier transform

X(j) = eat ejt dt
0
1 (a+j)t
= e
a + j 0
1
=
a + j
which is complex, and in terms of a magnitude and phase function is
1
|X(j)| = (i)
a2 + 2

1
X(j) = tan (ii)
a
Other examples are given in the class handout.
1.3 Properties of the Fourier Transform

The properties of the Fourier transform are covered more fully in the class handout and are
simply summarized here:
(1) Existence of the Fourier Transform The three Dirichlet conditions are
sucient conditions, but are not strictly necessary:
The function x(t) must be integrable in the absolute sense over all time,
that is
|x(t)| dt < .

45
There must be at most a nite number of maxima and minima in the
function x(t). Notice that periodic functions are excluded by this and
the previous condition.
There must be at most a nite number of discontinuities in the function
x(t), and all such discontinuities must be nite in magnitude.
(2) Linearity of the Fourier Transform If
F F
g(t) G(j) and h(t) H(j)
then for arbitrary constants a and b,
F
ag(t) + bh(t) aG(j) + bH(j). (3)
(3) Duality
F
X(jt) 2x()
where X(jt) is X(j) where has been replaced by t, and x() is x(t)
where t is replaced by . Therefore if we know the Fourier transform of
one function, we also know it for another.
(4) Even and Odd Functions
The Fourier transform of a real even function of time is a real even
function
The Fourier transform of an real odd function is an imaginary odd
function.
The same relationships hold for the inverse Fourier transform.
(5) Time Shifting The Fourier transform of x(t + ), a time shifted version
of x(t), is
F{x(t + )} = ej X(j).
and in terms of a magnitude and phase

X(j)+ )
F{x(t + )} = |X(j)| ej( .
(4) Time Scaling
F 1
x(at) X (j/a) , a = 0
|a|
(9) Time Reversal If a = 1, the time scaling property gives
F
x(t) X(j). (4)
(5) Waveform Energy Parsevals theorem asserts the equivalence of the total
waveform energy in the time and frequency domains by the relationship

2 1 2 1
|x(t)| dt = |X(j)| d = X(j)X(j)d.
2 2
In other words, the quantity |X(j)|2 is a measure of the energy of the
function per unit bandwidth.
46
(6) The Fourier Transform of the Derivative of a Function

dx
F = jX(j),
dt
The Fourier transform of the nth derivative of x(t) is
d x
F n
= (j)n X(j) (5)
dt
(7) The Fourier Transform of the Integral of a Function

t
1
F x()d = X(0)(j) + X(j)
j
(7) Time Reversal If a function x(t) has a Fourier transform X(j) then
F {x(t)} = X(j).
1.4 Extension of the Fourier Transform to Functions for which the Fourier
Integral does not Converge.
The Dirichlet conditions are sucient but not necessary conditions for the existence of
the Fourier transform. If the use of the Dirac delta function (x) is allowed, the Fourier
transform of many functions with a non-convergent Fourier integral may de dened. This
topic is covered in greater detail in the class handout (Sec. 4.4), and a simple example is
given here
Example 4
Dene the Fourier transform of the unit-step (Heaviside) function us (t), where

0 t<0
us (t) =
1 t 0.
Clearly the Fourier integral

jt
Us (j) = us (t)e dt = ejt dt
0
does not converge, and we seek an alternative approach.

Consider the one-sided real exponential function
x(t) = us (t)eat
47
as described in Example 3, and note that us (t) = lima0 x(t) which implies
Us (j) = lima0 X(j). From Example 3
1 a
X(j) = = 2 2
j 2
a + j a + a + 2
The real part is in the form of a Cauchy distribution, and we note that for a > 0

a
2 2
d =
a +
and that as a 0, a/(a2 + 2 ) becomes impulse-like, Therefore, as a 0,

a 1
2 2
() and j 2 2
j
a + a +
so that we may dene the Fourier transform of the unit-step function as
1
Us (j) = () +
j
2 The Frequency Response of a Linear System Dened Directly

from the Fourier Transform
The system frequency response function H(j) may be dened directly using the trans
form property of derivatives. Consider a linear system described by the single input/output
dierential equation
dn y dn1 y dy
an n
+ an1 n1
+ . . . + a1 + a0 y =
dt dt dt
dm u dm1 u du
bm m + bm1 m1 + . . . + b1 + b0 u
dt dt dt
and assume that the Fourier transforms of both the input u(t) and the output y(t) exist.
Then the Fourier transform of both sides of the dierential equation may be found by using
the derivative property: n
d f
F = (j)n F (j)
dtn
to give

an (j)n + an1 (j)n1 + . . . + a1 (j) + a0 Y (j) =

bm (j)m + bm1 (j)m1 + . . . + b1 (j) + b0 U (j),
which has reduced the original dierential equation to an algebraic equation in j. This
equation may be rewritten explicitly in terms of Y (j) in terms of the frequency response
H(j)
bm (j)m + bm1 (j)m1 + . . . + b1 (j) + b0
Y (j) = U (j)
an (j)n + an1 (j)n1 + . . . + a1 (j) + a0
= H(j)U (j),
showing the multiplicative frequency domain relationship between input and output.
48
c o n v o lu tio n
tim e d o m a in u (t) y (t)

F re q u e n c y R e s p o n s e
F
H ( jW ) F -1
F o u r ie r d o m a in U ( jW ) Y ( jW ) = U ( jW ) H ( jW )
m u ltip lic a tio n
3 Relationship between the Frequency Response and the Impulse

Response
The Dirac delta function (t) has a unique property; its Fourier transform is unity for all
frequencies
F {(t)} = 1,
The impulse response of a system h(t) is dened to be the response to an input u(t) = (t),
the output spectrum is then Y (j) = F {h(t)},
Y (j) =
F {(t)} H(j)
= H(j).
or
h(t) = F 1 {H(j)} .
In other words, the system impulse response h(t) and its frequency response H(j) are a
Fourier transform pair:
F
h(t) H(j).
In the same sense that H(j) completely characterizes a linear system in the frequency
response, the impulse response provides a complete system characterization in the time
domain.
4 The Convolution Property

A system with an impulse response h(t), driven by an input u(t), responds with an output
y(t) given by the convolution integral
y(t) = h(t) u(t)

= u( )h(t )d

In the frequency domain the input/output relationship for a linear system is multiplicative,
that is Y (j) = U (j)H(j). Because by denition
F
y(t) Y (j),
49
we are lead to the conclusion that
F
h(t) u(t) H(j)U (j). (6)
The computationally intensive operation of computing the convolution integral has been
replaced by the operation of multiplication. This result, known as the convolution property
of the Fourier transform, can be shown to be true for the product of any two spectra, for
example F (j) and G(j)

j
F (j)G(j) = f ()e d. g( )ej d

= f ()g( )ej(+ ) d d,

and with the substitution t = +

F (j)G(j) = f (t )g( )d ejt dt

= (f (t) g(t)) ejt dt

= F {f (t) g(t)} .
A dual property holds: if any two functions, f (t) and g(t), are multiplied together in the
time domain, then the Fourier transform of their product is a convolution of their spectra.
The dual convolution/multiplication properties are
F
f (t) g(t) F (j)G(j) (7)
F 1
f (t)g(t) F (j) G(j). (8)
2
410
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 51
Reading:
Class handout: The Laplace Transform.
Class handout: Understanding Poles and Zeros.
Class handout: Sinusoidal Frequency Response of Linear Systems.
1 The One-Sided Laplace Transform

Consider a causal waveform x(t), such as the unit-step function us (t), for which the Fourier
integral
jt
x(t)e dt = x(t)ejt dt
0
does not converge. Clearly, for us (t), the Fourier integral

Us (j) = 1.ejt dt
0
does not converge.

Now consider a modied function x(t) = x(t)w(t) where w(t) is a weighting function with
the property limt w(t) = 0, chosen to ensure convergence, so that

X (j, w) = x(t)w(t)ejt dt
0
may be considered an approximation to X(j).

In particular, consider w(t) = et for real , and note that as 0, x(t)w(t) x(t),
so that the Fourier transform is

t
t jt
F x(t)e = X(j, ) = x(t)e e dt = x(t)e(+j)t dt.
0 0
If we dene a complex variable s = + j we can write

L {x(t)} = X(s) = x(t)est dt
0
which denes the one-sided Laplace transform. (See the handout for the denition of the
two-sided transform).
1 c D.Rowell 2008
copyright
51
The Laplace transform may be considered as an extension of the Fourier transform
(for causal functions) that includes an exponential weighting term to extend the range
of functions for which the integral converges.
Note that for causal waveforms
X(j) = X(s)|s=j
and if x(t) is non-causal
X(s)|s=j ,
X(j) =
for example F {sin(0 t)} = L {sin(0 t) }|s=j , since the Laplace transform assumes x(t) 0
for t < 0 .
f(t) f(t)
0 t 0 t
n o n - c a u s a l s in u s o id c a u s a l s in u s o id
Example 1
The following are some simple examples of Laplace transforms:

1
L {us (t) } = 1.est dt =
s
0
L {(t) } = (t)est dt = 1
0

at 1
L e = e(s+a)t dt =
0 s+a
1.1 The Derivative Property of the Laplace Transform:

If a function x(t) has a Laplace transform X(s), the Laplace transform of the derivative of
x(t) is
dx
L = sX(s) x(0).
dt
Using integration by parts

dx dx st
L = e dt
dt 0 dt

= x(t)est 0 + sx(t)est dt
0
= sX(s) x(0).
52
This procedure may be repeated to nd the Laplace transform of higher order derivatives,
for example the Laplace transform of the second derivative is
2
dx dx
L = s [sL {x(t)} x(0)]
dt2 dt
t=0
dx
= s2 X(s) sx(0)
dt t=0
which may be generalized to

n
dn x di1 x
L = sn X(s) sni
dtn i=1
dti1 t=0
for the n derivative of x(t).
2 The Transfer Function

The use of the derivative property of the Laplace transform generates a direct algebraic
solution method for determining the response of a system described by a linear input/output
dierential equation. Consider an nth order linear system, completely relaxed at time t = 0,
and described by
dn y dn1 y dy
an n
+ a n1 n1
+ . . . + a1 + a0 y =
dt dt dt
dm u dm1 u du
bm m + bm1 m1 + . . . + b1 + b0 u.
dt dt dt
In addition assume that the input function u(t), and all of its derivatives are zero at time
t = 0, and that any discontinuities occur at time t = 0+ . Under these conditions the Laplace
transforms of the derivatives of both the input and output simplify to
n n
d y n d u
L = s Y (s), and L = sn U (s)
dtn dtn
so that if the Laplace transform of both sides is taken

n
an s + an1 sn1 + . . . + a1 s + a0 Y (s) =

bm sm + bm1 sm1 + . . . + b1 s + b0 U (s)
which has had the eect of reducing the original dierential equation into an algebraic
equation in the complex variable s. This equation may be rewritten to dene the Laplace
transform of the output:
bm sm + bm1 sm1 + . . . + b1 s + b0
Y (s) = U (s)
an sn + an1 sn1 + . . . + a1 s + a0
= H(s)U (s)
53
The Laplace transform generalizes the denition of the transfer function to a complete in
put/output description of the system for any input u(t) that has a Laplace transform.
The system response y(t) = L1 {Y (s)} may be found by decomposing the expression for
Y (s) = U (s)H(s) into a sum of recognizable components using the method of partial fractions
as described above, and using tables of Laplace transform pairs to nd the component
time domain responses. To summarize, the Laplace transform method for determining the
response of a system to an input u(t) consists of the following steps:
(1) If the transfer function is not available it may be computed by taking the
Laplace transform of the dierential equation and solving the resulting al
gebraic equation for Y (s).
(2) Take the Laplace transform of the input.
(3) Form the product Y (s) = H(s)U (s).
(4) Find y(t) by using the method of partial fractions to compute the inverse
Laplace transform of Y (s).
tim e d o m a in u (t)
y (t)
T r a n s fe r fu n c tio n
L H (s ) -1
L
L a p la c e d o m a in U (s ) Y ( s ) = U ( s ) H ( s )
Example 2
Determine the transfer function of the rst-order RC lter:
R
+
v (t) v (t)
in C o
-
The dierential equation relating vo (t) to vin (t) is

dv0
RC + vo = vin (t)
dt
and taking the Laplace transform of both sides gives
(RCs + 1)Vo (s) = Vin (s)
from which
Vo (s) 1
H(s) = =
Vin (s) RCs + 1
54
2.1 The Transfer function and the Sinusoidal Frequency Response

We have seen that
bm sm + bm1 sm1 + . . . + b1 s + b0
H(s) =
an sn + an1 sn1 + . . . + a1 s + a0
and
bm (j)m + bm1 (j)m1 + . . . + b1 (j) + b0
H(j) =
an (j)n + an1 (j)n1 + . . . + a1 (j) + a0
so that
H(j) = H(s)|s=j
3 Poles and Zeros of the Transfer Function

The transfer function provides a basis for determining important system response character
istics without solving the complete dierential equation. As dened, the transfer function is
a rational function in the complex variable s = + j, that is
bm sm + bm1 sm1 + . . . + b1 s + b0
H(s) =
an sn + an1 sn1 + . . . + a1 s + a0
It is often convenient to factor the polynomials in the numerator and denominator, and to
write the transfer function in terms of those factors:
N (s) (s z1 )(s z2 ) . . . (s zm1 )(s zm )
H(s) = =K ,
D(s) (s p1 )(s p2 ) . . . (s pn1 )(s pn )
where the numerator and denominator polynomials, N (s) and D(s), have real coecients
dened by the systems dierential equation and K = bm /an . The zi s are the roots of the
equation
N (s) = 0,
and are dened to be the system zeros, and the pi s are the roots of the equation
D(s) = 0,
and are dened to be the system poles. Clearly when s = zi the numerator N (s) = 0 and
the transfer function vanishes, that is
lim H(s) = 0.
szi
and similarly when s = pi the denominator polynomial D(s) = 0 and the value of the transfer
function becomes unbounded,
lim H(s) = .
spi
All of the coecients of polynomials N (s) and D(s) are real, therefore the poles and zeros
must be either purely real, or appear in complex conjugate pairs. In general for the poles,
either pi = i , or else pi , pi+1 = i ji . The existence of a single complex pole without a
corresponding conjugate pole would generate complex coecients in the polynomial D(s).
Similarly, the system zeros are either real or appear in complex conjugate pairs.
55
3.1 The Pole-Zero Plot
A system is characterized by its poles and zeros in the sense that they allow reconstruction of
the input/output dierential equation. In general, the poles and zeros of a transfer function
may be complex, and the system dynamics may be represented graphically by plotting their
locations on the complex s-plane, whose axes represent the real and imaginary parts of the
complex variable s. Such plots are known as pole-zero plots. It is usual to mark a zero
location by a circle () and a pole location a cross (). The location of the poles and zeros
provide qualitative insights into the response characteristics of a system. Many computer
programs are available to determine the poles and zeros of a system. The gure below is an
example of a pole-zero plot for a third-order system with a single real zero, a real pole and
a complex conjugate pole pair, that is;
(3s + 6) (s (2))
H(s) = 3 =3
(s + 3s2 + 7s + 5) (s (1))(s (1 2j))(s (1 + 2j))
p o le
z e ro s - p la n e
j2
0
-2 -1
- j2
Note that the pole-zero plot characterizes the system, except for the overall gain constant
K.
4 Frequency Response and the Pole-Zero Plot

The frequency response may be written in terms of the system poles and zeros by substituting
directly into the factored form of the transfer function:
(j z1 )(j z2 ) . . . (j zm1 )(j zm )
H(j) = K .
(j p1 )(j p2 ) . . . (j pn1 )(j pn )
Because the frequency response is the transfer function evaluated on the imaginary axis of
the s-plane, that is when s = j, the graphical method for evaluating the transfer function
may be applied directly to the frequency response. Each of the vectors from the n system
poles to a test point s = j has a magnitude and an angle:

|j pi | = i2 + ( i )2 ,

1 i
(s pi ) = tan ,
i
56
jW jW
jW
jW s - p la n e q s - p la n e
1 1
1
r 2
p q 1 r1
2 |
p 1 q
- p
1 2
f
1
1
|jW
f
0 s s
0
z 1
( jW - p ) q 2
1 2
p
p 2
2
as shown above, with similar expressions for the vectors from the m zeros. The magnitude
and phase angle of the complete frequency response may then be written in terms of the
magnitudes and angles of these component vectors
m
|(j zi )|
|H(j)| = K ni=1
i=1 |(j pi )|
m n
H(j) = (j zi ) (j pi ).
i=1 i=1
If the vector from the pole pi to the point s = j has length qi and an angle i from the
horizontal, and the vector from the zero zi to the point j has a length ri and an angle i ,
the value of the frequency response at the point j is
r1 . . . rm
|H(j)| = K (1)
q1 . . . qn
H(j) = (1 + . . . + m ) (1 + . . . + n ) (2)
The graphical method can be very useful for deriving a qualitative picture of a system
frequency response. For example, consider the sinusoidal response of a rst-order system
with a pole on the real axis at s = 1/ as shown below. Even though the gain constant
K cannot be determined from the pole-zero plot, the following observations may be made
directly by noting the behavior of the magnitude and angle of the vector from the pole to
the imaginary axis as the input frequency is varied:
jW |H ( jW ) |
K
K /q 1
jW 2
s - p la n e
K /q
2
q
2
0
W
q q 1 jW 1 H ( jW ) W 1 W 2
2
q 1 0
W
0 s
p = - 1 / t
-q 1
-q 2
-9 0
(a ) (b )
57
1. At low frequencies the gain approaches a nite value, and the phase angle has a small
but nite lag.
2. As the input frequency is increased the gain decreases (because the length of the vector
increases), and the phase lag also increases (the angle of the vector becomes larger).
3. At very high input frequencies the gain approaches zero, and the phase angle ap
proaches /2.
As a second example consider a second-order system, with the damping ratio chosen so that
the pair of complex conjugate poles are located close to the imaginary axis as shown below.
In this case there are a pair of vectors connecting the two poles to the imaginary axis, and
the following conclusions may be drawn by noting how the lengths and angles of the vectors
change as the test frequency moves up the imaginary axis:
|H ( jW ) |
jW
j W 2
p
1
W
j W 1 0
H ( jW ) W 1 W 2
0 s W
0
p
2
-1 8 0
( a ) (b )
1. At low frequencies there is a nite (but undetermined) gain and a small but nite
phase lag associated with the system.
2. As the input frequency is increased and the test point on the imaginary axis approaches
the pole, one of the vectors (associated with the pole in the second quadrant) decreases
in length and at some point reaches a minimum. There is an increase in the value of
the magnitude function over a range of frequencies close to the pole.
3. At very high frequencies, the lengths of both vectors tend to innity, and the magnitude
of the frequency response tends to zero, while the phase approaches an angle of
radians because the angle of each vector approaches /2.
The following generalizations may be made about the sinusoidal frequency response of a
linear system, based upon the geometric interpretation of the pole-zero plot:
1. If a system has an excess of poles over the number of zeros (n > m) the magnitude
of the frequency response tends to zero as the frequency becomes large. Similarly,
if a system has an excess of zeros (n < m) the gain increases without bound as the
58
frequency of the input increases. (This cannot happen in physical energetic systems
because it implies an innite power gain through the system.) If n = m the system
gain becomes constant at high frequencies.
2. If a system has a pair of complex conjugate poles close to the imaginary axis, the
magnitude of the frequency response has a peak, or resonance, at frequencies in the
proximity of the pole. If the pole pair lies directly upon the imaginary axis, the system
exhibits an innite gain at that frequency.
3. If a system has a pair of complex conjugate zeros close to the imaginary axis, the
frequency response has a dip or notch in its magnitude function at frequencies in
the vicinity of the zero. Should the pair of zeros lie directly upon the imaginary axis,
the response is identically zero at the frequency of the zero, and the system does not
respond at all to sinusoidal excitation at that frequency.
4. A pole at the origin of the s-plane (corresponding to a pure integration term in the
transfer function) implies an innite gain at zero frequency.
5. Similarly a zero at the origin of the s-plane (corresponding to a pure dierentiation)

implies a zero gain for the system at zero frequency.
59
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 61
Reading:
Class handout: Sinusoidal Frequency Response of Linear Systems (Sec. 7).
Class handout: Sinusoidal Frequency Response of Linear Systems (Sec. 6.1).
Class handout: Introduction to Continuous Time Filter Design.
1 Poles and Zeros of Filter Classes

We can use the s-plane relationship between a lters poles and zeros and the frequency
response function to make some general comments about the desgn of various classes of
lters:
(a) Low-Pass Filters:
To ensure a high-frequency roll-o the number of poles must exceed the number
of zeros, ie n > m. (In many low-pass lters m = 0.)
To ensure a nite low frequency gain there can be no poles or zeros at the origin.
| H ( j W ) |
jW s - p la n e
o n >
m
1
n o p o le s o r z e r o s a t th e o r ig in
x x
x s
x x
W
0
(b) High-Pass Filters
To ensure a constant high-frequency gain the number of poles must equal the
number of zeros, ie n = m.
To ensure a low frequency gain that approaches zero, there must be one or more
zeros at the origin.
1 c D.Rowell 2008
copyright
61
| H ( j W ) |
jW s - p la n e
n = m
1 n z e r o s a t th e o r ig in
x
x oo s
x
W
0 W c
(c) Band-Pass Filters
To ensure a high-frequency roll-o the number of poles must exceed the number
of zeros, ie n > m.
To ensure a low frequency gain that approaches zero, there must be one or more
zeros at the origin.
The band-pass characteristic is shaped by a group of poles clustered near the
imaginary axis in the region of the passband,
| H ( j W ) |
jW s - p la n e
x n > m
x
x p o le s c lu s te r e d n e a r th e
im a g in a r y a x is in r e g io n
o f th e p a s s -p a n d
1
o n e o r m o re z e r o s a t th e o r ig in
oo s
x
x
x
W
0 W c
W c 2
1
(d) Band-Stop Filters
To ensure a constant high-frequency gain the number of poles must equal the
number of zeros, ie n = m.
To ensure a nite low frequency gain there can be no poles or zeros at the origin.
The band-reject characteristic is shaped by a group of zeros clustered on or near
the imaginary axis in the region of the stopband,
62
|H ( jW ) |
jW s - p la n e
o n = m
o z e r o s c lu s te r e d o n o r n e a r th e
o
im a g in a r y a x is in r e g io n
o f th e s to p -p a n d
1
x x n o p o le s o r z e r o s a t th e o r ig in
x x s
x x
o
o
o
W
0 W r W r
1 2
2 The Decibel
Filter frequency response magnitudes |H(j)| are frequently plotted using the decibel log
arithmic scale. The Bel, named after Alexander Graham Bell, is dened as the logarithm
to base 10 of the ratio of two power levels. In practice the Bel is too large a unit, and the
decibel (abbreviated dB), dened to be one tenth of a Bel, has become the standard unit of
logarithmic power ratio. The power ow P into any element in a system, may be expressed
in terms of a logarithmic ratio Q to a reference power level Pref :

P P
Q = log10 Bel or Q = 10 log10 dB. (1)
Pref Pref
Because the power dissipated in a Dtype element is proportional to the square of the
amplitude of a system variable applied to it, when the ratio of across or through variables is
computed the denition becomes
2
A A
Q = 10 log10 = 20 log10 dB. (2)
Aref Aref
where A and Aref are amplitudes of variables. 2 Table ?? expresses some commonly used
decibel values in terms of the power and amplitude ratios.
The magnitude of the frequency response function |H (j)| is dened as the ratio of the
amplitude of a sinusoidal output variable to the amplitude of a sinusoidal input variable.
This ratio is expressed in decibels, that is
|Y (j)|
20 log10 |H(j)| = 20 log10 dB.
|U (j)|
As noted this usage is not strictly correct because the frequency response function does not
dene a power ratio, and the decibel is a dimensionless unit whereas |H (j)| may have
physical units.
2
This denition is only strictly correct when the two amplitude quantities are measured across a common
Dtype (dissipative) element. Through common usage, however, the decibel has been eectively redened
to be simply a convenient logarithmic measure of amplitude ratio of any two variables. This practice is
widespread in texts and references on system dynamics and control system theory. In this book we have also
adopted this convention.
63
Decibels Power Ratio Amplitude Ratio
-40 0.0001 0.01
-20 0.01 0.1
-10 0.1 0.3162
-6 0.25 0.5
-3 0.5 0.7071
0 1.0 1.0
3 2.0 1.414
6 4.0 2.0
10 10.0 3.162
20 100.0 10.0
40 10000.0 100.0
Table 1: Common Decibel quantities and their corresponding power and amplitude ratios.
3 Low-Pass Filter Design

The prototype low-pass lter is based upon the magnitude-squared of the frequency response
function |H(j)|2 , or the frequency response power function. The phase response of the lter
is not considered. We begin by dening tolerance regions on the power frequency response
plot, as shown below:.
| H ( jW ) |2
1 + e
1 + l 2
0
0 W c W r W (ra d /s e c )
p a s s b a n d tr a n s itio n b a n d s to p b a n d
The lter specications are that

1
1 |H(j)|2 > for || < c
1 + 2
1
and |H(j)|2 < for || > r ,
1 + 2
where c is the cut-o frequency, r is the rejection frequency, and and are design
parameters that select the lter attenuation at the two critical frequencies. For example, if
64
= 1, at c the power response response |H(jc )|2 = 0.5, the -3 dB response frequency.
In general we expect the response function to be monotonically decreasing in the transition
band.
The lter functions examined in this document will be of the form
|H(j)|2 = . (3)
1 + f 2 ()
where f () 0 as 0, and f () as to generate a low-pass lter action.
3.1 The Butterworth Filter

The Butterworth lter is dened by the power gain
1
|H(j)|2 =
1+ 2 (/c )2N
where N is a positive integer dening the lter order. Note that does not appear in this
formulation, but clearly N and are interrelated, since at = r
1 1

1+ 2
1 + (r /c )2N
2
which may be solved to show

log(/)
N
log(r /c )
endequation
The power response function of Butterworth lters for N = 1 . . . 5 is shown below:
|H(j)| 2 1
0.9
Power response
0.8
0.7
0.6
0.5
0.4
0.3
N increasing
0.2 N=1
0.1
N=5
0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Normalized frequency / c
Butterworth lters are also known as maximally at lters because the response has the
maximum number of vanishing derivatives at = 0 and = for lters of the form of Eq.
1.
65
3.1.1 The Butterworth Filter Transfer Function
The poles of the power gain transfer function may be found by substituting s = j in the
characteristic equation:
2N
2 s
1+ =0
jc
and solving for S, which yields 2N roots pv (n = 1 . . . 2N ) that lie on a circle with radius
r = c 1/N , and angular separation of /N rad:
pn = c 1/N ej(2n+N 1)/2N n = 1 . . . 2N
Notice that if N is odd a pair of poles will lie on the real axis at s = c 1/N , while if N
is even the roots will form complex conjugate pairs.
The six poles of |H(s)|2 for a third-order (N = 3) Butterworth lter are shown in (a)
below:
jW
s - p la n e jW
s - p la n e
n = 1
x
n = 1
r = W c e -1 /N x n = 6
r = W e -1 /N x
c
2 p /6 p /3
n = 2
x x s x s
n = 2 n = 5
x x
n = 4 n = 3 x
n = 3
(a ) (b )
For a stable causal system we must have

|H(j)|2 = H(j)H(j) = H(s)H(s)|s=j
with all poles in the left-half-plane, which allows us to take take the N poles of |H(j)|2 in
the left half-plane as the poles of the lter H(s), that is the poles specied by (n = 1 . . . N )
in Eq. (8) above,
pn = c 1/N ej(2n+N 1)/2N n = 1...N (4)
as is shown in (b) above.
If the lter is to have unity gain at low frequencies
lim |H(j)| = 1
0
we require the complete Butterworth transfer function to be

(p1 )(p2 ) . . . (pN )
H(s) =
(s p1 )(s p2 ) . . . (s pN )
(1)N p1 p2 . . . pN
= ,
(s p1 )(s p2 ) . . . (s pN )
where only the N stable left-half-plane poles (Eq. 4) are included.
66
Design Procedure:
1. Determine the lter order

log (/)
N
log (r /c )
2. Determine the Butterworth radius
r = c 1/N
3. Determine The Butterworth angles
2n + N 1
n = n = 1 . . . 2N
2N
4. Determine the N left half-plane poles
pn = rejn n = 1, . . . , N
5. Form the transfer function

p1 p2 . . . pN
H(s) =
(s p1 )(s p2 ) . . . (s pN )
67
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 71
Reading:
1 Butterworth Filter Design Example

(Same problem as in the Class Handout). Design a Butterworth low-pass lter to meet the
power gain specications shown below:
| H ( jW ) |2
1
0 .9
0 .0 5
0
0 1 0 2 0 W
At the two critical frequencies

1
= 0.9 = 0.3333
1 + 2
1
= 0.05 = 4.358
1 + 2
Then
log(/)
N = 3.70
log(r /c )
1 c D.Rowell 2008
copyright
71
we therefore select N=4. The 4 poles (p1 . . . p4 ) lie on a circle of radius r = c 1/N = 13.16
and are given by
|pn | = 13.16
pn = (2n + 3)/8
for n = 1 . . . 4, giving a pair of complex conjugate pole pairs
p1,4 = 5.04 j12.16

p2,3 = 12.16 j5.04
The transfer function, normalized to unity gain, is

29993
H(s) =
(s2 + 10.07s + 173.2)(s2 + 24.32s + 173.2)
and the lter Bode plots are shown below.

Bode Diagram
50
0
Magnitude (dB)
50
100
150
0
90
Phase (deg)
180
270
360
1 0 1 2 3
10 10 10 10 10
Frequency (rad/sec)
2 Chebyshev Filters
The order of a lter required to met a low-pass specication may often be reduced by relaxing
the requirement of a monotonically decreasing power gain with frequency, and allowing
72
ripple to occur in either the pass-band or the stop-band. The Chebyshev lters allow
these conditions:
1
Type 1 |H(j)|2 = (1)
1+ 2 TN2 (/c )
1
Type 2 |H(j)|2 = (2)
1+ 2 (TN2 (r /c )/TN2 (r /))
Where TN (x) is the Chebyshev polynomial of degree N . Note the similarity of the form
of the Type 1 power gain (Eq. (1)) to that of the Butterworth lter, where the function
TN (/c ) has replaced (/c )N . The Type 1 lter produces an all-pole design with slightly
dierent pole placement from the Butterworth lters, allowing resonant peaks in the pass
band to introduce ripple, while the Type 2 lter introduces a set of zeros on the imaginary
axis above r , causing a ripple in the stop-band.
The Chebyshev polynomials are dened recursively as follows
T0 (x) = 1
T1 (x) = x
T2 (x) = 2x2 1
T3 (x) = 4x3 3x
..
.
TN (x) = 2xTN 1 (x) TN 2 (x), N >1 (3)
with alternate denitions
TN (x) = cos(N cos1 (x)) (4)

= cosh(N cosh1 (x)) (5)
The Chebyshev polynomials have the min-max property:
Of all polynomials of degree N with leading coecient equal to one, the polynomial
TN (x)/2N 1
has the smallest magnitude in the interval |x| 1. This minimum maximum
amplitude is 21N .
In low-pass lters given by Eqs. (13) and (14), this property translates to the following
characteristics:
Filter Pass-Band Characteristic Stop-Band Characteristic
Butterworth Maximally at Maximally at
Chebyshev Type 1 Ripple between 1 and 1/(1 + 2 ) Maximally at
Chebyshev Type 2 Maximally at Ripple between 1 and 1/(1 + 2 )
73
2.1 The Chebyshev Type 1 Filter

With the power response from Eq. (13)
1
|H(j)|2 =
1+ 2 TN2 (/c )
and the lter specication from Fig. 1, the required lter order may be found as follows. At
the edge of the stop-band = r
1 1
|H(jr |2 =
1+ 2 TN2 (r /c ) 1 + 2
so that
TN (r /c ) = cosh N cosh1 (r /c )
and solving for N
cosh1 (/)
N (6)
cosh1 (r /c )
The characteristic equation of the power transfer function is

2 2 s s j
1 + TN = 0 or TN =
jc jc
Now TN (x) = cos(N cos1 (x)), so that

1 s j
cos N cos = (7)
jc
If we write cos1 (s/jc ) = + j, then
s = c (j cos ( + j))
= c (sinh sin + j cosh cos ) (8)
which denes an ellipse of width 2c sinh() and height 2c cosh() in the s-plane. The
poles will lie on this ellipse. Substituting into Eq. (16)

s
TN = cos (N ( + j))
jc
= cos N cosh N j sin N sinh N ,
the characteristic equation becomes

j
cos N cosh N j sin N sinh N = . (9)

0 for real x we require
Equating the real and imaginary parts in Eq. (21), (1) since cosh x =
cos N = 0, or
(2n 1)
n = n = 1, . . . , 2N (10)
2N
74
and, (2) since at these values of , sin N = 1 we have
1 1
= sinh1 (11)
N
As in the Butterworth design procedure, we select the left half-plane poles as the poles of
the lter frequency response.
Design Procedure:
1. Determine the lter order

cosh1 (/)
N
cosh1 (r /c )
2. Determine
1 1
= sinh1
N
3. Determine n , n = 1 . . . N
(2n 1)
n = n = 1, . . . , N
2N
4. Determine the N left half-plane poles
pn = c (sinh sin n + j cosh cos n ) n = 1, . . . , N
5. Form the transfer function
(a) If N is odd
p1 p2 . . . pN
H(s) =
(s p1 )(s p2 ) . . . (s pN )
(b) If N is even
1 p1 p2 . . . pN
H(s) = 2
1 + (s p1 )(s p2 ) . . . (s pN )
The dierence in the gain constants in the two cases arises because of
the ripple in the pass-band. When N is odd, the response |H(j0)|2 = 1,
whereas if N is even the value of |H(j0)|2 = 1/(1 + 2 ).
75
Example 1
Repeat the previous Butterworth design example using a Chebyshev Type 1
design.
From the previous example we have c = 10 rad/s., r = 20 rad/s., = 0.3333,
= 4.358. The required order is
cosh1 (/) cosh1 13.07

N = = 2.47
cosh1 (r /c ) cosh1 2
Therefore take N = 3. Determine :

1 1 1 1
= sinh = sinh1 (3) = 0.6061
N 3
and sinh = 0.6438, and cosh = 1.189. Also, n = (2n 1)/6 for n = 1 . . . 6
as follows:
n: 1 2 3 4 5 6
n : /6 /2 5/6 7/6 3/2 11/6
sin n : 1/2 1 1/2
1/2
-1 1/2

cos n : 3/2 0 3/2 3/2 0 3/2
Then the poles are
pn = c (sinh sin n + j cosh cos n )

1 3
p1 = 10 0.6438 + j1.189 = 3.219 + j10.30
2 2
p2 = 10 (0.6438 1 + j1.189 0) = 6.438

1 3
p3 = 10 0.6438 j1.189 = 3.219 j10.30
2 2

1 3
p4 = 10 0.6438 j1.189 = 3.219 j10.30
2 2
p5 = 10 (0.6438 0 j1.189 0) = 6.438

1 3
p6 = 10 0.6438 + j1.189 = 3.219 + j10.30
2 2
and the gain adjusted transfer function of the resulting Type 1 lter is
750
H(s) =
(s2 + 6.438s + 116.5)(s + 6.438)
The pole-zero plot for the Chebyshev Type 1 lter is shown below.
76
jW
X j1 0 .3 0 s - p la n e
X
-6 .4 3 8 -3 .2 1 9 0 s
X - j1 0 .3 0
2.2 The Chebyshev Type 2 Filter

The Chebyshev Type 2 lter has a monotonically decreasing magnitude function in the pass
band, but introduces equi-amplitude ripple in the stop-band by the inclusion of system zeros
on the imaginary axis. The Type 2 lter is dened by the power gain function:
1
|H(j)|2 = T 2 ( /c )
(12)
1+ 2 TN 2 (rr /)
If we make the substitutions

r c 1
= and =
TN (r /c )
Eq. 24 may be written in terms of the modied frequency
2 TN2 (/c )
|H(j)|2 = (13)
1 + 2 TN2 (/c )
which has a denominator similar to the Type 1 lter, but has a numerator that contains a
Chebyshev polynomial, and is of order 2N . We can use a method similar to that used in the
Type 1 lter design to nd the poles as follows:
1. First dene a complex variable, say = + j (analogous to the Laplace variable

s = + j used in the type 1 design) and write the power transfer function:
2 TN2 ( /jc )
|H( )|2 =
1 + 2 TN2 ( /jc )
77
The poles are found using the method developed for the Type 1 lter, the zeros are
found as the roots of the polynomial TN ( /jc ) on the imaginary axis = j. From
the denition TN (x) = cos (N cos1 (x)) it is easy to see that the roots of the Chebyshev
polynomial occur at

(n 1/2)
x = cos n = 1...N
N
and from Eq. (25) the system zeros will be at

(n 1/2)
n = jc cos n = 1 . . . N.
N
2. The poles and zeros are mapped back to the s-plane using s = r c / and the N left
half-plane poles are selected as the poles of the lter.
3. The transfer function is formed and the system gain is adjusted to unity at = 0.
Example 2
Repeat the previous Chebyshev Type 1 design example using a Chebyshev Type
2 lter.
From the previous example we have c = 10 rad/s., r = 20 rad/s., = 1/3,
= 4.358. The procedure to nd the required order is the same as before, and
we conclude that N = 3. Next, dene
r c 200
= =

1 3
= = = 0.1154
TN (r /c ) T3 (2)
Determine :

1 1 1
= sinh1 = sinh1 (8.666) = 0.9520
N 3
and sinh = 1.1024, and cosh = 1.4884.
The values of n = (2n 1)/6 for n = 1 . . . 6 are the same as the design for the
78
Type 1 lter, so that the poles of |H( )|2 are
pn = c (sinh sin n + j cosh cos n )

1 3
1 = 10 1.1024 + j1.4884 = 5.512 + j12.890
2 2
2 = 10 (1.1024 1 + j1.4884 0) = 11.024

1 3
3 = 10 1.1024 j1.488 = 5.512 j12.890
2 2

1 3
4 = 10 1.1024 j1.4884 = 5.512 j12.890
2 2

1
5 = 10 1.1024
j1.488 0 = 11.024
2

1 3
6 = 10 1.1024 + j1.4884 = 5.512 + j12.890
2 2
The three left half-plane poles (4 , 5 , 6 ) are mapped back to the s-plane using
s = r c / giving three lter poles
p1 , p2 = 5.609 j13.117
p3 = 18.14
The system zeros are the roots of
T3 (/jc ) = 4(/jc )3 3(/jc ) = 0
from the denition of TN (x), giving 1 = 0 and 2 , 3 = j8.666. Mapping these

back to the s-plane gives two nite zeros z1 , z2 = j23.07, z3 = (the zero at
does not aect the system response) and the unity gain transfer function is
p1 p2 p3 (s z1 )(s z2 )
H(s) =
z1 z2 (s p1 )(s p2 )(s p3 )
6.9365(s2 + 532.2)
=
(s + 18.14)(s2 + 11.22s + 203.5)
The pole-zero plot for this lter is shown in below. Note that the poles again lie
on ellipse, and the presence of the zeros in the stop-band.
79
jW
o j2 3 .0 7 s to p -b a n d
s - p la n e
j2 0
X j1 3 .1 2
j1 0
p a s s -b a n d
X
-5 .6 1 0 s
-1 8 .1 4
X - j1 3 .1 2
o - j2 3 .0 7
2.3 Comparison of Filter Responses

Bode plot responses for the three previous example lters are shown below:
50
0 Chebyshev Type 2 (N=3)

Magnitude (dB)
50
Chebyshev Type 1 (N=3)

100
Butterworth (N=4)
150
360

Phase (deg)

Butterworth (N=4)
360
0 1 2 3
10 10 10 10
Angular frequency (rad/sec)
710
While all lters meet the design specication, it can be seen that the Butterworth and
the Chebyshev Type 1 lters are all-pole designs and have an asymptotic high-frequency
magnitude slope of 20N dB/decade, in this case -80 dB/decade for the Butterworth design
and -60 dB/decade for the Chebyshev Type 1 design. The Type 2 Chebyshev design has
two nite zeros on the imaginary axis at a frequency of 23.07 rad/s, forcing the response to
zero at this frequency, but with the result that its asymptotic high frequency response has
a slope of only -20 dB/decade. Note also the singularity in the phase response of the Type
2 Chebyshev lter, caused by the two purely imaginary zeros.
The pass-band and stop-band power responses are shown in below. Notice that the design
method developed here guarantees that the response will meet the specication at the cut-o
frequency (in this case |H(j)|2 = 0.9 at c = 10. Other design methods (such as used by
MATLAB) may not use this criterion.
1.02
Butterworth (N=4)
1
Power response
0.98
0.96
0.94
0.92
0.9 Chebyshev Type 1 (N=3)

0.88
0 1 2 3 4 5 6 7 8 9 10
0.035
0.03
Power response
0.025
Butterworth (N=4)
0.02
0.015 Chebyshev Type 1 (N=3)

0.01
0.005
0
20 30 40 50 60 70 80 90 100
711
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 81
Reading:
Class Handout: Introduction to Operational Ampliers.
Class Handout: Op-Amp Implementation of Analog Filters.
1 Second-Order Filter Sections

(a) Low-Pass Filter
a0
Hlp (s) =
s2 + a1 s + a0
High Frequency Behavior: The number of poles exceeds the number of zeros (n
m = 2) so that
lim |H(j)| = 0.

and the high frequency asymptotic slope is -40dB/decade.

Low Frequency Behavior:
lim |H(j)| = 1
0

Mid Frequency Behavior: The response in the region a0 is determined by
the systems damping ratio , and will exhibit a resonant peak if < 0.707.
|H ( jW ) |
0 W
1 c D.Rowell 2008
copyright
81
(b) High-Pass Filter

s2
Hhp (s) =
s2 + a1 s + a0
High Frequency Behavior: The number of poles equals the number of zeros (n =
m) so that
lim |H(j)| = 1.

Low Frequency Behavior: There are a pair of zeros at the origin so that
lim |H(j)| = 0
0
and the low frequency asymptotic slope is +40dB/decade.

Mid Frequency Behavior: The response in the region a0 is determined by
the systems damping ratio , and will exhibit a resonant peak if < 0.707.
|H ( jW ) |
0
W
(c) Band-Pass Filter

a1 s
Hbp (s) =
s2 + a1 s + a0
High Frequency Behavior: The number of poles exceeds the number of zeros (n
m = 1) so that
lim |H(j)| = 0.

and the high frequency asymptotic slope is -20dB/decade.

Low Frequency Behavior: There is a single of zero at the origin so that
lim |H(j)| = 0
0
and the low frequency asymptotic slope is +20dB/decade.

Mid Frequency Behavior: When s = j a0 ,

H(j a0 ) = 1
which denes the passband center frequency.
82
|H ( jW ) |
0 W
a 0
(d) Band-Stop Filter

s2 + a0
Hbs (s) = 2
s + a1 s + a0
High Frequency Behavior: The number of poles equals the number of zeros (n =
m = 2) so that
lim |H(j)| = 1.

Low Frequency Behavior: There are no zeros at the origin and
lim |H(j)| = 1

Mid Frequency Behavior: There are a pair of imaginary zeros at s = j a0 forcing

the response magnitude to zero at a frequency = a0 .

|H(j a0 )| = 0
which denes the band rejection (notch) center frequency.
|H ( jW ) |
0 W
a 0
83
2 Transformation of Low-Pass Filters to other Classes

Filter specication tolerance bands for high-pass, band-pass and band-stop lters are shown
The most common procedure for the design of these lters is to design a prototype low-pass
lter using the methods described above, and then to transform the low-pass lter to the
desired form by a substitution in the transfer function, that is we substitute a function
g(s) for s in the low-pass transfer function Hlp (s), so that the new transfer function is
H (s) = Hlp (g(s)). The eect is to modify the lter poles and zeros to produce the desired
frequency response characteristic.
| H ( jW ) |2
R c
R s
0 W r W c W (ra d /s e c )
( a ) H ig h - p a s s
| H ( jW ) |2
R c
R s
W W W
0
0 rl c l W c u ru W (ra d /s e c )
(b ) B a n d -p a s s
| H ( jW ) |2
R c
R s
0 W c l W rl W ru W c u W (ra d /s e c )
(c ) B a n d -s to p
The critical frequencies used in the design are as shown above. For band-pass and band-stop
lters it is convenient to dene a center frequency o as the geometric mean of the pass-pand
84
edges, and a bandwidth :

o = cu cl
= cu cl .
The transformation formulas for a low-pass lter with cut-o frequency c are given below
c 1
Low-pass (c1 ) Low-pass (c2 ) g(s) =
s
c 2
2
Low-pass (c ) High-pass (c ) g(s) = c
s
s2 + 2o
Low-pass (c = ) Band-pass (cl , cu ) g(s) =
s
s2c
Low-pass (c = ) Band-stop (cl , cu ) g(s) = 2
s + 2o
The band-pass and band-stop transformations both double the order of the lter, since s2
is involved it the transformation. The low-pass lter is designed to have a cut-o frequency
c = cu cl .
The above transformations will create an ideal gain characteristic from an ideal low-
pass lter. For practical lters, however, the skirts of the pass-bands will be a warped
representation of the low-pass prototype lter. This does not usually cause problems.
Example 1
Transform the rst-order low-pass lter
c
Hlp (s) =
s + c
to a high-pass lter Hhp (s).
Using the transformation g(s) = 2c /s
s
Hhp (s) = c =
2c s + c
s
+ c
Example 2
Transform the second-order low-pass Butterworth lter
2c
Hlp (s) =
s2 + 2c s + 2c
85
to a high-pass lter Hhp (s).
Using the transformation g(s) = 2c /s

2c s2
Hhp (s) = =
2 2
c 2
+ 2
sc + c2 s2 + 2s + c
s
Example 3
Design a second-order bandpass lter with center frequency o and bandwidth
.
Step 1: Design a rst-order prototype low-pass lter with cut-o frequency
:

Hlp (s) =
s +
Step 2: Transform the prototype using
s2 + 2o
g(s) =
s
so that
s
H(s) = =
s2 +2o
+ s2 + s + 2o
s
Example 4
Design a second-order band-stop lter with center frequency o and notch-width

.
Step 1: Design a rst-order prototype low-pass lter with cut-o frequency
:
Hlp (s) =
s +
Step 2: Transform the prototype using
s2
g(s) =
s2 + 2o
so that
s2 + 2
o
H(s) = =
s2
+ s2 + s + 2o
s2 +2o
86
3 State-Variable Active Filters

Practical realizations of analog lters are usually based on factoring the transfer function
into cascaded second-order sections, each based on a complex conjugate pole-pair or a pair
of real poles, and a rst-order section if the order is odd. Any zeros in the system may be
distributed among the second- and rst-order sections. Each rst- and second-order section
is then implemented by an active lter and connected in series. For example the third-order
Butterworth high-pass lter
s3
H(s) =
s3 + 40s2 + 800s + 8000
would be implemented as
s2 s
H(s) = 2

s + 20s + 400 s + 20
as shown below:
2
s 3
s s
s 3
+ 4 0 s 2
+ 8 0 0 s + 8 0 0 0
s 2
+ 2 0 s + 4 0 0 s + 2 0
(a ) (b )
The design of each low-order block can be handled independently.

The state-variable lter design method is based on the block diagram representation used
in the so-called phase-variable description of linear systems that uses the outputs of a chain
of cascaded integrators as state variables. Consider a second-order lter block with a transfer
function
Y (s) b2 s2 + b1 s + b0
H(s) = = 2
U (s) s + a1 s + a0
and split H(s) into two sub-blocks representing the denominator and numerator by intro
ducing an intermediate variable x and rewrite
X(s) 1
H1 (s) = = 2
U (s) s + a1 s + a0
Y (s)
H2 (s) = = b2 s2 + b1 s + b0
X(s)
so that H(s) = H2 (s)H1 (s).
The dierential equations corresponding to H1 (s) and H2 (s) are
d2 x dx
2
+ a1 + a0 x = u
dt dt
and
d2 x dx
y = b2 2
+ b1 + b0 x.
dt dt
87
The rst equation may be rewritten explicitly in terms of the highest derivative
d2 x dx
= a 1 a0 x + u. (1)
dt2 dt
Consider a pair of cascaded analog integrators with the output dened as x(t) so that the
derivatives of x(t) appear as inputs to the integrators:
2
d x d x
2
d t 1 d t 1
s x (t)
s
Note that Eq. (1) gives an explicit expression for the input to the rst block in terms of
the outputs of the two integrators and the system input, and therefore generates the block
diagram for H1 (s) shown below:
2
d x d x
2
+ d t 1 d t 1
u (t) x (t)
s s
- -
a a 0
1
The equation
d2 x dx
y = b2 2 + b1 + b0 x.
dt dt
shows that the output y(t) is a weighted sum of x(t) and its derivatives, leading to the
complete second-order state variable lter block diagram:
+ y (t)
+ + +
b 2 b 1 b 0
+ 1 1
u (t) x (t)
s s
- - d 2 x d x
d t 2 d t
a a 0
1
This basic structure may be used to realize the four basic lter types by appropriate
choice of the numerator.
Y1 (s) a0
Hlp (s) = = a unity gain low-pass lter (2)
U (s) s2 + a1 s + a0
Y2 (s) a1 s
Hbp (s) = = 2
a unity gain band-pass lter (3)
U (s) s + a1 s + a0
Y3 (s) s2
Hhp (s) = = a unity gain high-pass lter (4)
U (s) s2 + a1 s + a0
Y4 (s) s2 + a0
Hbs (s) = = a unity gain band-stop lter (5)
U (s) s2 + a1 s + a0
88
+
a 0
Y 4 (s ) (b a n d -s to p )
+
Y 3 ( s ) ( h ig h - p a s s )
a 1 Y 2 (s ) (b a n d -p a s s )
+ 1 1
U (s ) a 0 Y 1 ( s ) ( lo w - p a s s )
s s
- s 2
X (s ) s X (s ) X (s )
-
a a 0
1
89
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 91
Reading:
Class Handout: Introduction to the Operational Amplier
Class Handout: Op-amp Implementation of Analog Filters
1 Operational-Amplier Based State-Variable Filters

We saw in Lecture 8 that second-order lters may be implemented using the block diagram
structure
+
a 0
Y 4 (s ) (b a n d -s to p )
+
Y 3 ( s ) ( h ig h - p a s s )
a 1 Y 2 (s ) (b a n d -p a s s )
+ 1 1
U (s ) a 0 Y 1 ( s ) ( lo w - p a s s )
s s
- s 2
X (s ) s X (s ) X (s )
-
a a 0
1
and that a high-order lter may be implemented by cascading second-order blocks, and
possibly a rst-order block (if the lter order is odd).
We now look into a method for implementing this lter structure using operational am
pliers.
1.1 The Operational Amplier

What is an operational amplier? It is simply a very high gain electronic amplier, with a
pair of dierential inputs. Its functionality comes about through the use of feedback around
the amplier, as we show below.
G a in : A
v + +
v o u t
v - -
1 c D.Rowell 2008
copyright
91
The op-amp has the following characteristics:
It is basically a three terminal amplier, with two inputs and an output. It is a

dierential amplier, that is the output is proportional to the dierence in the voltages
applied to the two inputs, with very high gain A,
vout = A(v+ v )
where A is typically 104 105 , and the two inputs are known as the non-inverting (v+ )
and inverting (v ) inputs respectively. In the ideal op-amp we assume that the gain A
is innite.
In an ideal op-amp no current ows into either input, that is they are voltage-controlled
and have innite input resistance. In a practical op-amp the input current is in the
order of pico-amps (1012 ) amp, or less.
The output acts as a voltage source, that is it can be modeled as a Thevenin source
with a very low source resistance.
The following are some common op-amp circuit congurations that are applicable to the
active lter design method described here. (See the class handout for other common cong
urations).
The Inverting Amplier:
s u m m i n g j u n c t i o n R f
( v ir tu a l g o u n d )
R in if
v in -
v
i in + o u t
In the conguration shown above we note
Because the gain A is very large, the voltage at the node designated summing junc
tion is very small, and we approximate it as v = 0 the so-called virtual ground
assumption.
We assume that the current i into the inverting input is zero.
Applying Kirchos Current law at the summing junction we have

vin vo
i1 + if = + =0
R1 Rf
from which
Rf
vout = vin
Rin
92
The voltage gain is therefore dened by the ratio of the two resistors. The term inverting
amplier comes about because of the sign change.
The Inverting Summer: The inverting amplier may be extended to include multiple
inputs:
s u m m in g ju n c tio n
( v ir tu a l g o u n d ) R f
R 1
v
1 if
i1
-
R 2 v
v 2 + o u t
i2
As before we assume that the inverting input is at a virtual ground (v 0) and apply
Kirchos current law at the summing junction
v1 v2 Vout
i1 + i2 + if = + + =0
R1 R2 Rf
or
Rf Rf
vout = v1 + v2
R1 R2
which is a weighted sum of the inputs.
The summer may be extended to include several inputs by simply adding additional input
resistors Ri , in which case
n
Rf
vout = vi
i=1
R i
The Integrator: If the feedback resistor in the inverting amplier is replaced by a capac
itor C the amplier becomes an integrator.
s u m m in g ju n c tio n C
( v ir tu a l g o u n d )
R i n
if
v
i n -
v
i in + o u t
At the summing junction we apply Kirchos current law as before but the feedback current
is now dened by the elemental relationship for the capacitor:
vin dvout
iin + if = +C =0
Rin dt
93
Then
dvout 1
= vin
dt Rin C
or
t
1
vout = vin dt + vout (0)
Rin C 0
As above, the integrator may be extended to a summing conguration by simply adding

extra input resistors:
n
1 t vi
vout = dt + vout (0)
C 0 i=1 Ri
and if all input resistors have the same value R
t n

1
vout = vi dt + vout (0)
RC 0 i=1
1.2 A Three Op-Amp Second-Order State Variable Filter

A conguration using three op-amps to implement low-pass, high-pass, and bandbass lters
directly is shown below:
h ig h - p a s s o u tp u t
R 3
R 4 C 1
C 2
- R 1
R R
5 - v 2
+
V 1
in v - v
+
3 2
A lo w - p a s s o u tp u t
3
A +
1
R A 2
6
b a n d -p a s s o u tp u t
Ampliers A1 and A2 are integrators with transfer functions

1 1 1 1
H1 (s) = and H2 (s) = .
R1 C1 s R 2 C2 s
Let 1 = R1 C1 and 2 = R2 C2 . Because of the gain factors in the integrators and the sign
inversions we have
dv2 d2 v2
v1 (t) = 2 and v3 (t) = 1 2 2 .
dt dt
Amplier A3 is the summer. However, because of the sign inversions in the op-amp circuits
we cannot use the elementary summer conguration described above. Applying Kirchos
Current Law at the non-inverting and inverting inputs of A3 gives
Vin v+ v1 v+ v3 v v2 v
+ = 0 and + = 0.
R5 R6 R4 R1
94
Using the innite gain approximation for the op-amp, we set v = v+ and
R3 R5 R4 R6
v3 v1 + v2 = Vin ,
R3 + R4 R5 + R6 R3 + R4 R5 + R6
and substituting for v1 and v3 we generate a dierential equation in v2

d2 v2 1 + R4 /R3 dv2 R4 1 1 + R4 /R3
+ + v2 = Vin
dt2 1 (1 + R6 /R5 ) dt R 3 1 2 1 2 (1 + R5 /R6 )
which corresponds to a low-pass transfer function with

Klp a0
H(s) =
s2 + a1 s + a0
where

R4 1
a0 =
R 1 2
3
1 + R4 /R3 1
a1 =
1 + R6 /R5 1
1 + R3 /R4
Klp =
1 + R5 /R6
A Band-Pass Filter: Selection of the output as the output of integrator A1 generates

the transfer function
Kbp a1 s
Hbp (s) = 1 sHlp (s) =
s2 + a1 s + a0
where
R6
Kbp =
R5
A High-Pass Filter: Selection of the output as the output of the summer A3 generates
the transfer function
Khp s2
Hhp (s) = 1 2 s2 Hlp (s) =
s2 + a 1 s + a 0
where
1 + R4 /R3
Khp =
1 + R5 /R6
A Band-Stop Filter: The band-stop conguration may be implemented with an addi
tional summer to add the outputs of ampliers A2 and A (with appropriate weights).
1.3 A Simplied Two Op-amp Based State-variable Filter:

If the required lter does not require a high-pass action (that is, access to the output of
the summer A1 above) the summing operation may be included at the input of the rst
integrator, leading to a simplied circuit using only two op-amps shown below:
95
C 1
C 2
R 1
V - V R 2
1
-
in
+
R 3 + V o u t
A 1
A 2
Consider the input stage:
if C 1
(a ) v
R 1
-
a
V in
i1 V
R
i3 + 1
3
A 1
V o u t
With the innite gain assumption for the op-amps, that is V = V+ , and with the assumption
that no current ows in either input, we can apply Kirchos Current Law (KCL) at the
node designated (a) above:
1 1
i1 + if i3 = (Vin va ) + sC1 (v1 va ) va =0
R1 R3
Assuming va = Vout , and realizing that the second stage is a classical op-amp integrator with
transfer function
Vout (s) 1
=
v1 (s) R 2 C2 s
1 1
(Vin Vout ) + sC1 (R2 C2 sVout Vout ) Vout =0
R1 R3
which may be rearranged to give the second-order transfer function
Vout (s) 1/1 2

= 2
Vin (s) s + (1/2 )s + (1 + R1 /R3 )/1 2
which is of the form

Klp a0
Hlp (s) =
s2 + a1 s + a0
where
1
a0 = (1 + R1 /R3 )
1 2
1
a1 =
2
1
Klp =
1 + R1 /R3
96
1.4 First-Order Filter Sections:
Single pole low-pass lter sections with a transfer function of the form
K0
H(s) =
s + 0
may be implemented in either an inverting or non-inverting conguration as shown in Fig.
11.
R 1
R 2 R 3
C
R 2 R -
1
V in - V in + V o u t
+ V o u t
C
(a ) (b )
The inverting conguration (a) has transfer function

Vout (s) Zf R1 1/R1 C
= =
Vin (s) Zin R2 s + 1/R1 C
where 0 = 1/R1 C and K = R1 /R2 .

The non-inverting conguration (b) is a rst-order R-C lag circuit buered by a non-
inverting (high input impedance) amplier (see the class handout) with a gain K = 1 +
R3 /R2 . Its transfer function is

Vout (s) R3 1/R1 C
= 1+ .
Vin (s) R2 s + 1/R1 C
97
Classroom Demonstration
Example 2 in the class handout Op-Amp Implementation of Analog Filters describes a

state-variable design for a 5th-order Chebyshev Type I low-pass lter with c = 1000 rad/s
and 1dB ripple in the passband.
The transfer function is
122828246505000
H(s) =
(s2
+ 468.4s + 429300)(s2 + 178.9s + 988300)(s + 289.5)
429300 988300 289.5
= 2 2
s + 468.4s + 429300 s + 178.9s + 988300 s + 289.5
which is implemented in the handout as a pair of second-order two-op-amp sections followed
by a rst-order block:
0 .4 7
0 .4 7
V in 4 6 4 2
- 4 5 4 2
-
4 6 4 2 +
+
7 7 0
0 .4 7
0 .4 7
- 1 1 8 9 3
-
7 7 0 +
+
1 5 0 0 4 5 0 0
7 3 4 9 -
+ V o u t
0 .4 7
This lter was constructed on a bread-board using 741 op-amps, and was demonstrated
to the class, driven by a sinusoidal function generator and with an oscilloscope to display
the output. The demonstration included showing (1) the approximately 10% ripple in the
passband, and (2) the rapid attenuation of inputs with frequency above 157 Hz (1000 rad/s).
98
2 Introduction to Discrete-Time Signal Processing

Consider a continuous function f (t) that is limited in extent, T1 t < T2 . In order to
process this function in the computer it must be sampled and represented by a nite set of
numbers. The most common sampling scheme is to use a xed sampling interval T and to
form a sequence of length N : {fn } (n = 0 . . . N 1), where
fn = f (T1 + nT ).
In subsequent processing the function f (t) is represented by the nite sequence {fn } and the
sampling interval T .
In practice, sampling occurs in the time domain by the use of an analog-digital (A/D)
converter.
D ig ita l S ig n a l
P ro c e s s o r
re c o n s tru c to r
s a m p le r
f(t) P ro c e s s e s d is c r e te s a m p le s y (t)
o f th e in p u t f n = f(n T ) to
p ro d u c e a d is c r e te o u tp u t
s e q u e n c e y n .
(i) The sampler (A/D converter) records the signal value at discrete times nT to produce
a sequence of samples {fn } where fn = f (nT ) (T is the sampling interval.
f(t)
t
0 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T
(ii) At each interval, the output sample yn is computed, based on the history of the input
and output, for example
1
yn = (fn + fn1 + fn2 )
3
3-point moving average lter, and
yn = 0.8yn1 + 0.2fn
is a simple recursive rst-order low-pass digital lter. Notice that they are algorithms.
(iii) The reconstructor takes each output sample and creates a continuous waveform.
In real-time signal processing the system operates in an innite loop:
99
f(t)
s a m p le f( t)
t
f
n
c o m p u te y
y (t) s ta ir c a s e o r n
z e ro -o rd e r
w a v e fo rm
re c o n s tru c t y (t)
0 t
0 tim e
2.1 Sampling
The mathematical operation of sampling (not to be confused with the operation of an analog-
digital converter) is most commonly described as a multiplicative operation, in which f (t) is
multiplied by a Dirac comb sampling function s(t; T ), consisting of a set of delayed Dirac
delta functions:

s(t; T ) = (t nT ).
n=

We denote the sampled waveform f (t) as

f (t) = s(t; T )f (t) = f (t)(t nT )
n=
f(t)
s (t;D T )
f * (t) = f(x )s (t;D T )
910
Note that f (t) is a set of delayed and weighted delta functions, and that the waveform must
be interpreted in the distribution sense by the strength (or area) of each component impulse.
The implied process to produce the discrete sample sequence {fn } is by integration across
each impulse, that is
nT + nT +

fn = f (t)dt = f (t)(t nT )dt
nT nT n=
or
fn = f (nT )
by the sifting property of (t).
2.2 The Spectrum of the Sampled Waveform f (t):

Notice that sampling comb function s(t; T ) is periodic and is therefore described by a
Fourier series:
1 jn0 t

s(t; T ) = e
T n=
where all the Fourier coecients are equal to (1/T ), and where 0 = 2/T is the fun
damental angular frequency. Using this form, the spectrum of the sampled waveform f (t)
may be written

jt 1
F (j) = f (t) e dt = f (t) ejn0 t ejt dt
T n=
1

= F (j ( n0 ))
T n=

1

2n
= F j
T n= T
The Fourier transform of a sampled function f (t) is periodic in the frequency domain with
period 0 = 2/T , and is a superposition of an innite number of shifted replicas of the
Fourier transform, F (j), of the original function scaled by a factor of 1/T .
F ( jW ) F * ( jW )
1 s a m p lin g a t D T
1 /D T
- p /D T 0 p /D T W - 4 p /D T 0 p /D T 2 p /D T 4 p /D T W
- 2 p /D T - p /D T
(a ) (b )
911
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 101
Reading:
Class Handout: Sampling and the Discrete Fourier Transform
Proakis & Manolakis (4th Ed.) Secs. 6.1 6.3, Sec. 7.1
Oppenheim, Schafer & Buck (2nd Ed.) Secs. 4.1 4.3, Secs. 8.1 8.5
1 The Sampling Theorem

Given a set of samples {fn } and its generating function f (t), an important question to ask
is whether the sample set uniquely denes the function that generated it? In other words,
given {fn } can we unambiguously reconstruct f (t)? The answer is clearly no, as shown
below, where there are obviously many functions that will generate the given set of samples.
f(t)
In fact there are an innity of candidate functions that will generate the same sample set.
The Nyquist sampling theorem places restrictions on the candidate functions and, if
satised, will uniquely dene the function that generated a given set of samples. The theorem
may be stated in many equivalent ways, we present three of them here to illustrate dierent
aspects of the theorem:
A function f (t), sampled at equal intervals T , can not be unambiguously

reconstructed from its sample set {fn } unless it is known a-priori that f (t)
contains no spectral energy at or above a frequency of /T radians/s.
In order to uniquely represent a function f (t) by a set of samples, the sampling

interval T must be suciently small to capture more than two samples per
cycle of the highest frequency component present in f (t).
There is only one function f (t) that is band-limited to below /T radians/s

that is satised by a given set of samples {fn }.
1 c D.Rowell 2008
copyright
101
Note that the sampling rate, Fs = 1/T , must be greater than twice the highest cyclic
frequency Fmax in f (t). Thus if the frequency content of f (t) is limited to max radians/s
(or Fmax cycles/s) the sampling interval T must be chosen so that

T <
max
or equivalently
1
T <
2Fmax
The minimum sampling rate to satisfy the sampling theorem FN = max / samples/s is
known as the Nyquist rate.
1.1 Aliasing
Consider a sinusoid
f (t) = A sin(at + )
sampled at intervals T , so that the sample set is
{fn } = {A sin(anT + )} ,
and noting that sin(t) = sin(t + 2k) for any integer k,

2m
fn = A sin(anT + ) = A sin a+ nT +
T
where m is an integer, giving the following important result:
Given a sampling interval of T , sinusoidal components with an angular frequency

a and a + 2m/T , for any integer m, will generate the same sample set.
In the gure below, a sinusoid is undersampled and a lower frequency sinusoid, shown as a
dashed line, also satises the sample set.
f(t)
o
o
T t
o
0
o
o
o
o
102
This phenomenon is known as aliasing. After sampling any spectral component in F (j)
above the Nyquist frequency /T will masquerade as a lower frequency component
within the reconstruction bandwidth, thus creating an erroneous reconstructed function.
The phenomenon is also known as frequency folding since the high frequency components
will be folded down into the assumed system bandwidth.
One-half of the sampling frequency (i.e. 1/(2T ) cycles/second, or /T radians/second)
is known as the aliasing frequency, or folding frequency for these reasons.
W
3 p /D T 5 p /D T
p /D T 3 p /D T
- p /D T 0 p /D T
- 3 p /D T - p /D T
- 5 p /D T - 3 p /D T
The following gure shows the eect of folding in another way. In (a) a function f (t) with
Fourier transform F (j ) has two disjoint spectral regions. The sampling interval T is
chosen so that the folding frequency /T falls between the two regions. The spectrum
of the sampled system between the limits /T < /T is shown in (b). The
frequency components above the aliasing frequency have been folded down into the region
/T < /T .
F ( jW ) F * ( jW )
s a m p lin g a t D T
a lia s e d s p e c tr a l a lia s e d s p e c tr a l
c o m p o n e n ts c o m p o n e n ts
W W
- p /D T 0 p /D T - p /D T 0 p /D T
(a ) (b )
1.2 Anti-Aliasing Filtering:

Once a sample set {fn } has been taken, there is nothing that can be done to eliminate the
eects of aliased frequency components. The only way to guarantee that the sample set
unambiguously represents the generating function is to ensure that the sampling theorem
criteria have been met, either by
103
1. Selecting a sampling interval T suciently small to capture all spectral components,
or
2. Processing the continuous-time function f (t) to eliminate all components at or above

the Nyquist rate.
The second method involves the use of a continuous-time processor before sampling f (t). A
low-pass aanti-aliasing lter is used to eliminate (or at least attenuate) spectral components
at or above the Nyquist frequency. Ideally the anti-aliasing lter would have a transfer
function
1 for || < /T
H(j ) =
0 otherwise,.
A n ti- a lia s in g ~ {fn }

f ( t )
f(t) S a m p le r
lo w - p a s s filte r
F ( j W )
H ( jW ) = 0 , |W |> p /T F ( jW ) H ( jW ) D T
c o n tin u o u s d o m a in d is c r e te d o m a in
In practice it is not possible to design a lter with such characteristics, and a more realistic
goal is to reduce the oending spectral components to insignicant levels, while maintaining
the delity of components below the folding frequency.
1.3 Reconstruction of a Function from its Sample Set

We saw in Lecture 9 that the spectrum F (j ) of a sampled function f (t) is innite in
extent and consists of a scaled periodic extension of F (j ) with a period of 2/T , i.e.

1

2n
F (j ) = F j .
T n= T
F ( jW ) F * ( jW )
1 s a m p lin g a t D T
1 /D T
- p /D T 0 p /D T W - 4 p /D T 0 p /D T 2 p /D T 4 p /D T W
- 2 p /D T - p /D T
(a ) (b )
If it is assumed that the sampling theorem was obeyed during sampling, the repetitions in
F (j ) will not overlap, and in fact f (t) will be entirely specied by a single period of F (j ).
Therefore to reconstruct f (t) we can pass f (t) through an ideal low-pass lter with transfer
function H(j ) that will retain spectral components in the range /T < < /T and
reject all other frequencies.
104
Id e a l r e c o n s tr u c tio n
lo w - p a s s filte r f(t)
f * (t) H ( jW )
F * ( jW ) F ( jW )
W
- p /D T p /D T
If the transfer function of the reconstruction lter is

T for || < /T
H(j ) =
0 otherwise,
in the absence of aliasing in f (t), that is no overlap between replications of F (j ) in F (j ),

the lter output will be
y(t) = F 1 {F (j )H(j )} = F 1 {F (j )} = f (t).
The lters impulse response h(t) is
sin (t/T )
h(t) = F 1 {H(j )} = ,
t/T
h (t)
t
-4 D T -3 D T -2 D T -D T 0 D T 2 D T 3 D T 4 D T
and note that the impulse response h(t) = 0 at times t = nT for n = 1, 2, 3, . . . (the
sampling times). The output of the reconstruction lter is the convolution of the input
function f (t) with the impulse response h(t),

f (t) = f (t) h(t) = h() f (t )(t nT ) d
n=

sin (/T )
= f (t )(t nT ) d
n=
/T

sin ((t nT )/T )

= f (nT ) ,
n=
(t nT )/T
105
or in the case of a nite data record of length N

N 1
sin ((t nT )/T )
f (t) = fn .
n=0
(t nT )/T
This is known as the cardinal (or Whittaker) reconstruction function. It is a superposition

of shifted sinc functions, with the important property that at t = nT , the reconstructed
function f (t) = fn . This can be seen by letting t = nT , in which case only the nth term
in the sum is nonzero. Between the sample points the interpolation is formed from the
sum of the sinc functions. The reconstruction is demonstrated below, where a sample set
(N = 13) with three nonzero samples is reconstructed. The individual sinc functions are
shown, together with the sum (dashed line). Notice how the zeros of the sinc functions fall
at the sample points.
o o
o o o o o o o o o o
0.3
2 The Discrete Fourier Transform (DFT)

We saw in Lecture 8 that the Fourier transform of the sampled waveform f (t) can be written
as a scaled periodic extension of F (j )

1

2n
F (j ) = F j
T n= T
We now look at a dierent formulation of F (j ). The Fourier transform of the sampled

function f (t)

j t
F (j ) = f (t) e dt = f (t)(t nT ) ej t dt
n=

= f (nT ) ej nT
n=
106
by reversing the order of integration and summation, and using the sifting property of (t).
We note:
F (j ) is a continuous function of , but is computed from the sample points f (nT )

in f (t).
We have shown that F (j ) is periodic in with period 0 = 2/T .
We now restrict ourselves to a nite, causal waveform f (t) in the interval 0 t < nT , so
that it has N samples, and let

N 1

F (j ) = f (nT ) ej nT
n=0
which is known as the Discrete-Time Fourier Transform (DTFT).

As a further restriction consider computing a nite set of N samples of F (j ) in a single
period, from = 0 to 2/T , that is at frequencies
2m
m = for m = 0, 1, 2, . . . , N 1
N T
F * ( j W ) N e q u i- s p a c e d s a m p le s
in o n e p e r io d o f F * ( jW )
- p /D T 0 p / D T 2 p /D T
and writing Fm = F (j m ) = F (j 2m/N T ), the DTFT becomes

N 1
Fm = fn ej 2mn/N for m = 0, 1, 2, . . . , N 1
n=0
where fn = f (nT ). This equation is known as the Discrete Fourier Transform (DFT) and
relates the sample set {fn } to a set of samples of its spectrum {Fm } both of length N .
The DFT can be inverted and the sample set {fn } recovered as follows:
1
N 1
fn = Fm ej 2mn/N for n = 0, 1, 2, . . . , N 1
N m=0
which is known as the inverse DFT (IDFT). These two equations together form the DFT
pair.
107
The DFT operations are a transform pair between two sequences {fn } and {Fm }.
The DFT expressions do not explicitly involve the sampling interval T or the sam
pled frequency interval = 2/(nT ).
Simple substitution into the formulas will show that both Fm and fn are periodic
with period N , that is fn+pN = fn and Fm+pN = Fm for any integer p.
The inverse transform is easily demonstrated:

N 1 N 1
N 1
1 j 2mn/N 1
fn = Fm e = fk ej 2mk/N ej 2mn/N
N m=0 N m=0 k=0
N
1 N
1
1
= fk ej 2m(nk)/N
N k=0 m=0
1
= (N fn ) = fn
N
since
N
1
N for n = k
ej 2m(nk)/N =
m=0
0 otherwise.
As in the continuous Fourier transform case, we adopt the notations
DFT
{fn } {Fm }
{Fm } = DFT {fn }
{fn } = IDFT {Fm }
to indicate DFT relationships.
108
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 111
Reading:
Class Handout: Sampling and the Discrete Fourier Transform
Class Handout: The Fast Fourier Transform
Proakis and Manolakis (4th Ed.) Ch. 7
Oppenheim, Schafer & Buck (2nd Ed.) Chs. 8 & 9
1 The Discrete Fourier Transform continued

In Lecture 10 the DFT pair associated with a sample set {fn } of length N was dened as

N 1
Fm = fn ej 2mn/N
n=0
1
N1
fn = Fm ej 2mn/N
N m=0
The value Fm was interpreted as F (j ) evaluated at = 2/N T .
1.1 Organization of the DFT

The N components in a DFT represent one period of a periodic spectrum. The rst N/2 lines
in the spectrum represent physical frequencies 0 . . . (/T ) radians/second. The components
in the upper half of the sequence, FN/2+1 . . . FN 1 , may be considered to be the negative
frequency components FN/2+1 . . . F1 in the spectrum. It is common to translate the upper
half of the record to the the left side of a plot to enhance the physical meaning.
Fm N e q u i- s p a c e d s a m p le s Fm
in o n e p e r io d o f F m
in te r p r e t to p N /2 v a lu e s
a s r e p r e s e n tin g n e g a tiv e
fr e q u e n c ie s in F * ( jW ) .
m
0 N -1 N /2 N -1 0 N /2 -1
1 c D.Rowell 2008
copyright
111
1.2 Spectral Resolution of the DFT
The DFT pair provide a transform relationship between a pair of (complex) data sets {fn }
and {Fm }, each of length N . If the sampling interval associated with {fn } is T units, the
record duration is
T = N T.
The frequency resolution, or line spacing, , in the DFT is
2 2 1
= = rad/s, or F = Hz. (1)
N T T T
and the frequency range spanned by the N lines in the DFT is
2 1
N = rad/s, or N F = Hz. (2)
T T
The sequence {Fm } represents both the positive and negative frequencies in a two-sided
spectrum. The highest (positive) frequency component in the spectrum is half of this range
(the Nyquist frequency), that is
1
max = rad/s, or Fmax = Hz. (3)
T 2T
We conclude therefore, that the resolution within the DFT depends on the duration T of
the data record, and the maximum frequency depends on the sampling interval T .
1.3 Properties of the Discrete Fourier Transform

Because the DFT is derived directly as a sampled continuous Fourier transform, it inherits
most of the properties of the Fourier transform. We repeat some of the important properties
here. In addition other properties are based on the assumed periodicity of {fn } and {Fm }:
1. Linearity: If {fn } and {gn } are both length N , and
DFT DFT
{fn } {Fm } and {gn } {Gm }
then
DFT
a {fn } + b {gn } a {Fm } + b {Gm }
2. Symmetry Properties of the DFT: If {fn } is a real-valued sequence then

{Fm } = F m
from which it follows that {{Fm }} is an even function of m and {{Fm }} is an odd
function of m. Similarly the magnitude of {Fm } is an even function, and the phase is
an odd function. In addition
DFT DFT
E {{fn }} {{Fm }} and O {{fn }} {{Fm }}
where E and O denote the even and odd parts respectively.
112
3. Shifting Properties: If
DFT
{fn } {Fm }
then
ejmn0 Fm
DFT
{fnn0 }
and DFT
ejm0 n fn {Fmm0 }
where n0 and m0 are constants.
4. Periodicity As noted above, both {fn } and {Fm } are periodic with period N .
Example 1
What is the eect of zero-padding (adding extra zero amplitude samples) to a
data record {fn } before taking the DFT?
fn o r ig in a l r e c o r d ( le n g th N ) fn
z e r o - p a d d e d r e c o r d ( le n g th N + M )
M a d d e d z e ro s
0
n
N -1 0 n
N -1 N + M -1
Assume that the original N samples encapsulate f (t), so that f (t) 0 for
T > N T . Then let FmN denote the DFT of the original length N record

N 1
FmN = fn ej 2mn/N
n=0
We note that when related back to the continuous time domain, the Nyquist
frequency is N = /T , and the line spacing is = 2/N T .
When we ad M zeros to the end of the record
1
N +M
N 1
FmN +M = fn ej 2mn/(N +M ) = fn ej 2mn/(N +M )
n=0 n=0
since the M extra data point do not contribute to the DFT sums. However
we now have N + M spectral samples with the same Nyquist frequency
N = /T , but with a new line spacing = 2/(N + M )T .
The result is that padding the data record with zeros has increased the spec
tral resolution of the DFT.
113
Note that if the number of added zeros is an integer multiple of the original
record length (M = kN , for k = 1, 2, 3, . . . ), the original DFT samples FmN will
be preserved and k 1 new samples will be interpolated between the original
samples.

kN 1
N 1
kN j 2mn/(kN )
Fm = fn e = fn ej 2mn/(kN )
n=0 n=0
so that FmN = kN
Fkm .
Example: The waveform below was sampled with T = 1 sec to give a length
16 record.
f(t)
4
0
0 5 1 0 1 5 t (s e c )
-1
-2
-3
-4
The data record length was then extended to a length of 256 by zero-padding
with 240 zeros, and the FFT was then computed again. The following plot,
demonstrating the interpolation in the DFT was generated from the MATLAB
fragment:
k = 0:7;
f16 = [cos(2.*pi*k/8) + 3*cos(4*pi*k/8)+ sin(6*pi*k/8) zeros(1,8)];
f256 = [f16 zeros(1,240)];
plot(0:255,abs(fft(f256)),o);
hold;
stem(0:16:255, abs(fft(f16)),filled);
114
F m
1 4
1 2
1 0
0 m
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0
The original 16 DFT points are shown as lled circles (produced by the stem()
function). The interpolation of 15 new values between these points can be easily
seen.
Example 2
What is the eect of adding M zeros to the center of a DFT record {Fm } before
taking the inverse DFT?
Fm Fm
o r ig in a l D F T r e c o r d ( le n g th N ) c e n te r - p a d d e d D F T r e c o r d ( le n g th N + M )
M a d d e d z e ro s
m m
0 N -1 0 N + M -1
We assume that no aliasing has taken place in the original sampling operation.
Lets answer this by considering what the DFT would have been if we had sampled
f (t) at a higher rate, for example lets consider twice the original rate.
Since no aliasing was present in the original record, there will ne no aliasing
at the higher rate.
115
The total duration T = N T = 2N (T /2) , therefore the line spacing
remains constant.
The Nyquist frequency N = /(T /2) has been doubled.
The spectrum has been scaled.
This argument says that doubling the sampling rate (halving the interval T )
has simply inserted N samples into the center of the DFT record {Fm }, and
scaled the data points. In general, inserting kN zeros into the center of a DFT
record and then taking the inverse DFT of the new record of length (k + 1)N , is
to interpolate k 1 samples between each of the original N data points (and to
scale the data set).
The following MATLAB fragment was used to demonstrate the interpolation.
i=0:7;
f8=cos(2*pi*(i-1)/8) + 3*cos(4*pi*(i-1)/8)+ sin(6*pi*(i-1)/8) ;
% Take the DFT
Fin = fft(f8);
% Pad the center of the DFT with zeros.
Fout = zeros(56,1);
Fout(1:4) = Fin(1:4);
Fout(61: 64) = Fin(5:8);
% Take the inverse DFT, scale, and preserve the real part
fout = real(ifft(Fout))*8;
plot(0:63,fout,o);
hold on;
stem(0:8:63,f8,filled);
A data record of length 8 is created, and the DFT is computed. The center of
this record is padded with zeros to form a length 64 DFT. The inverse DFT is
then computed and plotted. The results are shown below. The original length 8
data sequence is shown with lled circles, and the interpolated data points are
shown as open circles.
fn
0 n
0 1 0 2 0 3 0 4 0 5 0 6 0
-1
-2
-3
-4
116
2 The Fast Fourier Transform (FFT)

Although the DFT was known for many decades, its utility was severely limited because of
the computational burden. The calculation of the DFT of an input sequence of an N length
sequence {fn }

N 1
2mn
Fm = fn ej N , m = 0, . . . , N 1 (4)
n=0
requires N complex multiplications to compute each on the N values, Fm , for a total of N 2

multiplications. Early digital computers had neither xed-point nor oating-point hardware
multipliers, and multiplication was performed by binary shift-and-add software algorithms.
Multiplication was therefore a computationally expensive and time consuming operation,
rendering machine computation of the DFT impractical for common usage.
The FFT is a computationally ecient algorithm to compute the DFT, by eliminating
redundant complex multiplications. In this course we look at a single implementation - the
radix-2 FFT with decimation in time and input-bit-reversal. Many other algorithms exist.
We start by writing the DFT as

N 1
2mn

N 1
j
Fm = fn e N = fn WNmn , m = 0, . . . , N 1
n=0 n=0
where WN = ej 2/N . We also note the following periodic and symmetry properties of WN
k(N n)
WN = WNkn = WNkn (complex conjugate symmetry),
k(n+N ) n(k+N )
WNkn = WN = WN (periodicity in n and k),
nN/2
WNn = WN for n N/2.
The FFT recognizes that these properties render many of the N 2 complex multiplications in
the DFT redundant.
Assume that the input sequence length N is even. We then ask whether any compu
tational eciency might be gained from splitting the calculation of {Fm } into two sub-
computations, each of length N/2, involving the even samples, {f2n }, and odd samples
{f2n+1 } for n = 0 . . . N/2 1. Then

P 1
P 1
m(2n+1)
Fm = f2n WN2mn + f2n+1 WN
n=0 n=0

P 1
P 1
= f2n WN2mn + WNm f2n+1 WN2mn
n=0 n=0

P 1
P 1
= f2n WPmn + WNm f2n+1 WPmn
n=0 n=0
= Am + WNm Bm , m = 0, . . . , N 1.
117
where P = N/2, {Am } is a DFT of length N/2, based on the even sample points, and similarly
{Bm } is a DFT of length N/2 based on the odd sample points of {fn }. For example, if N = 8
the DFT relationship of {fn } shown as
fn N = 8 Fm
N = 8
p e r io d ic w ith p e r io d 8 p e r io d ic w ith p e r io d 8
D F T
n m
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
may decomposed into even and odd sample sets with a DFT relationship for the even set
indicated as
fn s e le c t th e e v e n s a m p le s o f { f n } to A
fo r m a n e w s e t o f le n g th 4 .
m
N = 4
p e r io d ic w ith p e r io d 4
D F T
n m
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
o n e p e r io d
We also note from the properties of the DFT that both {Am } and {Bm } are periodic
with period N/2, that is
Am+N/2 = Am , and Bm+N/2 = Bm
so that
Fm = Am + WNm Bm for m = 0 . . . (N/2 1), and
mN/2
Fm = AmN/2 WN BmN/2 for m = N/2 . . . (N 1)
These equations show that a DFT of length N may be synthesized by combining

two shorter DFTs from an even/odd decomposition of the original data set. For
example, if N = 8, F3 and F7 are simply related:
F3 = A3 + W83 B3
F7 = A7 + W87 B7 = A3 W83 B3
so that N/2 multiplications are required to combine the two sets. Each of the two
shorter DFTs requires (N/2)2 complex multiplications, therefore the total required
is N 2 /2 + N/2 < N 2 , for N > 2, indicating a computational saving.
A modied discrete form of Masons signal-ow graph is commonly used to display the
algorithmic structure of the synthesis. The gure below shows the signal-ow graph - con
sisting of a network of nodes connected by line segments.
118
a + W n b
a N
n
The algorithm works from left to right, and each right-hand node is assigned a value that
is the weighted sum of the connected left-hand nodes, where the indicated weight n is the
exponent of WN . If no weight is indicated, it is assumed to be unity (or equivalent to WN0 ).
Thus the output of the step shown above is c = a + WNn b.
With this notation the combining of the two length N/2 DFTs is illustrated below for
N=8. Each right-hand node is one of the Fm , formed by the appropriate combination of Am
and Bm .
0
A F0 = A + W B
4 - p o in t D F T 0 0 8 0
fr o m e v e n s a m p le s 0
{ f 0 , f 2 , f 4 , f6 } A 1 F1 = A 1 + W 8
1
B 1
1
A 2
2 F2 = A 2 + W 8 B 2
2
A 3
3 F3 = A 3 + W 8 B 3
3
4 0
B 4 F4 = A + W B = A - W B
4 - p o in t D F T 0 4 8 4 0 8 0
fr o m o d d s a m p le s
{ f 1 , f 3 , f 5 , f7 } B 5 F5 = A 5 + W 5
B 5 = A 1 - W 8
1
B 1
1 8
B 2 6 F6 = A + W 6
B 6 = A 2 - W 2
B
6 8 8 2
7 3
B 3 7 F7 = A 7 + W 8 B 7 = A 3 - W 8 B 3
This only the rst step in the development of the FFT. The complete algorithm will be
developed in Lecture 12.
119
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 121
Reading:
Class Handout: The Fast Fourier Transform
Proakis and Manolakis (4th Ed.): Secs. 8.1 8.3
Oppemheim Schafer & Buck (2nd Ed.): Secs. 9.0 9.3
1 The Fast Fourier Transform (contd.)

In Lecture 11 we saw that we could write the DFT of a length N sequence as

N 1
2mn

N 1
j
Fm = fn e N = fn WNmn , m = 0, . . . , N 1
n=0 n=0
where WN = ej 2/N . We noted that the number of complex multiplication operations

to compute the DFT is N 2 , but if we divided the original sequence into two length N/2
sequences (based on even and odd samples) and computed the DFT of each shorter sequence,
they could be combined
Fm = Am + WNm Bm for m = 0 . . . (N/2 1), and
mN/2
Fm = AmN/2 WN BmN/2 for m = N/2 . . . (N 1)
where {Am } is the DFT of the even-numbered samples, and {Bm } is the DFT of the odd-
numbered samples.
0
A F0 = A + W B
4 - p o in t D F T 0 0 8 0
fr o m e v e n s a m p le s 0
{ f 0 , f 2 , f 4 , f6 } A 1 F1 = A 1 + W 8
1
B 1
1
A 2
2 F2 = A 2 + W 8 B 2
2
A 3
3 F3 = A 3 + W 8 B 3
3
4 0
B 4 F4 = A + W B = A - W B
4 - p o in t D F T 0 4 8 4 0 8 0
fr o m o d d s a m p le s
{ f 1 , f 3 , f 5 , f7 } B 5 F5 = A 5 + W 5
B 5 = A 1 - W 8
1
B 1
1 8
B 2 6 F6 = A + W 6
B 6 = A 2 - W 2
B
6 8 8 2
7 3
B 3 7 F7 = A 7 + W 8 B 7 = A 3 - W 8 B 3
1 c D.Rowell 2008
copyright
01
The total number of required complex multiplications is (N/2)2 for each shorter DFT, and
N/2 to combine the two, giving a total of N (N + 1)/2, which is less than N 2 .
If N is divisible by 4, the process may be repeated, and each length N/2 DFT may be
formed by decimating the two N/2 sequences into even and odd components, forming the
length N/4 DFTs, and combining these back into a length N/2 DFT, as is shown for N = 8
below:
4 2 - p o in t D F T s 2 4 - p o in t D F T s 8 - p o in t D F T
2 - p o in t D F T F 0
fr o m s a m p le s 0 0
{ f 0 , f4 }
F 1
2 1
2 - p o in t D F T 4 F2
fr o m s a m p le s 2
{ f 2 , f6 }
6 F 3
3
2 - p o in t D F T 4 F 4
fr o m s a m p le s 0
{ f 1 , f5 }
5 F 5
2
2 - p o in t D F T 4 6 F 6
fr o m s a m p le s
{ f 3 , f7 }
6 7 F 7
Notice that all weights in the gure are expressed by convention as exponents of W8 .
In general, if the length of the data sequence is an integer power of 2, that is N = 2q for
integer q, the DFT sequence {Fm } may be formed by adding additional columns to the left
and halving the length of the DFT at each step, until the length is two. For example if
N = 256 = 28 a total of seven column operations would be required.
The nal step is to evaluate the N/2 length-2 DFTs. Each one may be written
F0 = f0 + W20 f1 = f0 + f1
F1 = f0 + W21 f1 = f0 f1 ,
which is simply the sum and dierence of the two sample points. No complex multiplications
are necessary. The 2-point DFT is shown in signal-ow graph form below, and is known as
the FFT buttery.
0
f0 F0 = f0 + W 2
f1 = f0 + f1
0
1
f 1 1 F 1 = f0 + W 2 f 1 = f0 - f1
The complete FFT algorithm for N = 8 is shown below. We note that if N = 2q , there
will be q = log2 (N ) columns in the signal-ow graph, and after the sum and dierence to
form the 2-point DFTs there will be log2 (N ) 1 column operations, each involving N/2
complex multiplications, giving a total of N/2 (log2 (N ) 1) N 2 . We will address the
issue of computational savings in more detail later.
02
8 - p o in t d a ta s e t 4 2 - p o in t D F T s 2 4 - p o in t D F T s 8 - p o in t D F T
f0 F 0
0 0 0
f 4 4 F 1
2 1
f 2 4 F2
0 2
f 6 4 6 F 3
3
f1 4 F 4
0 0
f5 4 5 F 5
2
f3 4 6 F 6
0
f 7 4 6 7 F 7
Input Bit-Reversal:
Notice that the algorithm described above requires that the input sequence {fn } be re
ordered in the left-hand column to accomplish the even-odd decomposition at each step.
The particular order reqired by this form of the FFT is known as input bit reversal, which
refers to a simple algorithm to determine the position k of the sample fn in the re-ordered
sequence:
1. Express the index n as a N -bit binary number.
2. Reverse the order of the binary digits (bits) in the binary number.
3. Translate the bit-reversed binary number back into decimal, to create the position in
the sequence k.
For example, the re-ordered position of f37 in a data sequence of length N = 256 = 28 is
found from
bit reversal
3710 = 001001012 101001002 = 16410
so that f37 would be positioned at k = 164 in the decimated input sequence.
The re-ordering procedure for N = 8 is:
Input position n: 0 1 2 3 4 5 6 7
(000)2 (001)2 (010)2 (011)2 (100)2 (101)2 (110)2 (111)2
Bit reversal
(000)2 (100)2 (010)2 (110)2 (001)2 (101)2 (011)2 (111)2
Modied position k: 0 4 2 6 1 5 3 7
The Inverse Fast Fourier Transform (IFFT):
The inverse FFT is dened as
1
N 1
2mn
fn = F m ej N , n = 0, . . . , N 1 (1)
N m=0
03
While the IFFT can be implemented in the same manner as the FFT described above, it is
possible to use a forward FFT routine to compute the IFFT as follows: Since the conjugate
of a product is the product of the conjugates, if we take the complex conjugate of both sides
we have
1
N 1
2mn
fn = F m ej N .
N m=0
The right-hand side is recognized as the DFT of F m and can be computed using a for
ward FFT, such as described above. The complete IDFT may therefore be computed by
conjugating the output, that is
N 1
1 2mn
fn = F m ej N , n = 0, . . . , N 1 (2)
N m=0
The steps are:
1. Conjugate the data set {Fm }.
2. Compute the forward FFT.
3. Conjugate the result and divide by N .
Computational Savings of the FFT:

As expressed above the computational requirements (in terms of complex multiplications) is
MFFT = (N/2) log2 (N ) if the initial 2-point DFTs are implemented with exponentials. The
number of complex multiplications for the direct DFT computation is MDFT = N 2 ) We can
therefore dene a speed improvement factor MFFT /MDFT as is shown below:
N MDFT MFFT MFFT /MDFT

4 16 4 0.25
8 64 12 0.188
16 256 32 0.125
32 1,024 80 0.0781
64 4,096 192 0.0469
128 16,384 448 0.0273
256 65,536 1024 0.0156
512 262,144 2,304 0.00879
1024 1,048,576 5,120 0.00488
2048 4,194,304 11,264 0.00268
4096 16,777,216 24,576 0.00146
2 Spectral Leakage in the DFT and Apodizing (Windowing) Func

tions
Often apparently spurious spectral components will appear in the output of a DFT compu
tation. This phenomenon is known as spectral leakage. We examine the origin of this eect
04
briey here by considering a nite sample set (length N ) of a sinusoid of the form
f (t) = cos(at)
so that
fn = cos(anT ) n = N/2, . . . N/2 1
Notice that in eect we have sampled a windowed version of f (t), or
f(t) = f (t)rect(N T )
where the rect function is dened

1 |t| < 1/2,
rect(t) =
0 otherwise
The Fourier transform of the windowed sinusoid is the convolution of the two Fourier trans
forms
F {cos(at)} = (( a) + ( + a))
sin(N T /2)
F {rect(N T )} = N T
N T /2
and the Fourier transform of the product is
F (j) = F {rect(N T ) cos(at)}
= (F {rect(N T )} F {cos(at)})
2
N T sin (( a)N T /2) sin (( + a)N T /2)
= +
2 ( a)N T /2 ( + a)N T /2
and the spectrum of the sampled waveform is
1
F (j) = F (j)
T
N sin (( a)N T /2) sin (( + a)N T /2)
= +
2 ( a)N T /2 ( + a)N T /2
which is a pair of sinc functions centered on frequencies = a and = a. The spacing of
the zeros of each of the sinc functions is at intervals of = 2/N T .
We may consider the DFT as a comb lter that displays discrete lines of the spectrum
F (j) at frequencies:
2m
= ; m = 0, 1, 2, . . . , N 1
N T
so that in the DFT,
2m
Fm = F
N T
Now consider what happens in two situations:
05
(a) The frequency a in f (t) = cos(at) is such that the data record contains an integer
number of periods. In this case the length of the data record
2 2M
N T = M or a= , for M integer.
a N T
The DFT is
N sin((m M )) sin((m + M ))
Fm = + (3)
2 (m M ) (m + M )
that is Fm = N/2 for m = M and Fm = 0 otherwise, which is what we would expect.
(b) The frequency a in f (t) = cos(at) is such that the data record does not contain an integer
number of periods. In this case the peak and zeros of the sinc functions will not line
up with the frequencies 2m/(N T ) in the DFT and the results will (1) not show the
peak of the sinc functions, and (2) will show nite components at all frequencies. This
is the origin of spectral leakage.
Spectral leakage occurs when frequency components in the input function f (t) are
not harmonics of the fundamental frequency k0 = 2/(N T ), dened by the length
of data record N T . Under such conditions the lines in the DFT do not accurately
reect the amplitude of the component, and spurious components appear adjacent
to the component.
This phenomenon is illustrated in the following two gures based on a DFT of length 16. In
the rst case the frequency of the the sinusoid is chosen so that there are four cycles in the
data record. The DFT shows two clean components at the appropriate frequency with no
evidence of leakage.
0.5
f(t)
0.5
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
t (sec)
10
8
Magnitude
0
8 6 4 2 0 2 4 6 8
m
In the second case the data record contains 3.5 cycles of the sinusoidal component. The
spectral leakage is severe: both the height of the main peak is reduced, and signicant
amplitudes are recorded for all spectral components.
06
1
0.5
f(t)
0
0.5
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
t (sec)
4
Magnitude
0
8 6 4 2 0 2 4 6 8
m
Reduction of Leakage by an Apodizing (Windowing) Function The reason for the

appearance of leakage components in the DFT of a truncated data set is the convolution of
the data spectrum with that of the truncation window (the rect function). Each sinusoidal
component in f (t) has a spectrum F (j) that is a pair of impulses in the frequency domain:
multiplication by the truncating function causes a spread in the width of the component.
Leakage may be reduced (but not eliminated) by using a dierent function to truncate
the series instead of the implicit rect function. These functions are known as an apodizing,
or windowing, functions and are chosen to smoothly taper the data record to zero at the
extremities, while minimizing the spectral spreading of each component. The data record
then becomes
f(t) = f (t)w(t)
or in the samples
fn = fn wn
where w(t) (or wn ) is the windowing function. There are many windowing functions in
common use, the following are perhaps the most common:
Bartlett Window: This is a triangular ramp, tapered to zero at the extremities of the
record

1 |t T /2| /(T /2) 0t<T
w(t) =
0 otherwise,

1 |n N/2| /(N/2) 0n<N
wn =
0 otherwise.
Hanning Window This is a smoothly tapered window

0.5 (1.0 + cos((t T /2)/(T /2))) 0t<T
w(t) =
0 otherwise,

0.5 (1.0 + cos((n N/2)/(N/2))) 0n<N
wn =
0 otherwise.
07
Hamming Window This is a smoothly tapered window that is similar to the Hanning
window

0.54 + 0.46 cos((t T /2)/(T /2)) 0t<T
w(t) =
0 otherwise.

0.54 + 0.46 cos((n N/2)/(N/2)) 0n<N
wn =
0 otherwise.
The magnitude spectra of these three windows are shown below, in linear and logarithmic
plots for (a) Bartlett, (b) Hanning, and (c) Hamming windows. In each case the spectrum
is shown in linear and logarithmic (dB) form. The frequency scale is normalized to units
of line spacing (2/N T ) radians/sec. The spectrum of the implicit rectangular window is
shown as a dotted line in each case. The various windows are a compromise and trade-o the
width of the central peak and attenuation of leakage components distant from the peak. For
example, the Hamming window has greater attenuation of components close to the cental
peak, while the Hanning window has greater attenuation away from the peak.
Magnitude Magnitude (dB)
1
0
Bartlett Bartlett
0.8
20
0.6
40
0.4
60
0.2
0 80
10 5 0 5 10 10 5 0 5 10
Frequency Frequency
(a)
1
0
Hanning Hanning
0.8
20
0.6
40
0.4
60
0.2
0 80
10 5 0 5 10 10 5 0 5 10
Frequency (b) Frequency
1 0
Hamming Hamming
0.8
20
0.6
40
0.4
60
0.2
0 80
10 5 0 5 10 10 5 0 5 10
Frequency Frequency
(c)
In each case it can be noted:
08
The width of the central lobe is wider than that of the rectangular window, indicating
that the main lobe will be approximately two lines in width.
The magnitude of the side-lobes is signicantly reduced, indicating that leakage away
from the main peak will be reduced.
These two eects are demonstrated in the following two plots. In the rst plot there are three
periods in the data record, and the data set as used has been windowed using a Hanning
function. In the non-windowed case there would be no leakage but the central peak has now
been smeared to occupy three lines.
0.5
f(t)
0.5
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
t (sec)
4
Magnitude
0
8 6 4 2 0 2 4 6 8
m
In the second case the data set contains 3.5 periods of a cosinusoid, and has been windowed
with a Hanning function. Here it can be seen that the leakage components away from the
main peak have been signicantly attenuated.
09
0.5
f(t)
0
0.5
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
t (sec)
3
Magnitude
0
8 6 4 2 0 2 4 6 8
m
010
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 131
Reading:
Proakis & Manolakis, Chapter 3 (The z-transform)
Oppenheim, Schafer & Buck, Chapter 3 (The z-transform)
1 Introduction to Time-Domain Digital Signal Processing

Consider a continuous-time lter
C o n t i n u o u s
f(t) s y s te m y ( t )
( h ( t ) , H ( s ) )
such as simple rst-order RC high-pass lter:

C
+
V R v o
-
described by a transfer function

RCs
H(s) = .
RCs + 1
The ODE describing the system is
dy df
+y =
dt dt
where = RC is the time constant.
Our task is to derive a simple discrete-time equivalent of this prototype lter based on
samples of the input f (t) taken at intervals T .
fn D S P y
a lg o r ith m n
1 c D.Rowell 2008
copyright
131
If we use a backwards-dierence numerical approximation to the derivatives, that is
dx (x(nT ) x((n 1)T )

dt T
and adopt the notation yn = y(nT ), and let a = /T ,
a(yn yn1 ) + yn = a(fn fn1 )
and solving for yn
a a a
yn = yn1 + fn fn1
1+a 1+a 1+a
which is a rst-order dierence equation, and is the computational formula for a sample-
by-sample implementation of digital high-pass lter derived from the continuous prototype
above. Note that
The delity of the approximation depends on T , and becomes more accurate when
T .
At each step the output is a linear combination of the present and/or past samples
of the output and input. This is a recursive system because the computation of the
current output depends on prior values of the output.
In general, regardless of the design method used, a LTI digital lter implementation will be
of a similar form, that is

N
M
yn = ai yni + bi fni
i=1 i=0
where the ai and bi are constant coecients. Then as in the simple example above, the
current output is a weighted combination of past values of the output, and current and past
values of the input.
If ai 0 for i = 1 . . . N , so that

M
yn = bi fni
i=0
The output is simply a weighted sum of the current and prior inputs. Such a lter is
a non-recursive lter with a nite-impulse-response (FIR), and is known as a moving
average (MA) lter, or an all-zero lter.
If bi 0 for i = 1 . . . M , so that

N
yn = ai yni + b0 fn
i=0
only the current input value is used. This lter is a recursive lter with an innite-
impulse-response (IIR), and is known as an auto-regressive (AR) lter, or an all-pole
lter.
132
With the full dierence equation

N
M
yn = ai yni + bi fni
i=1 i=0
the lter is a recursive lter with an innite-impulse response (IIR), and is known as
an auto-regressive moving-average (ARMA) lter.
2 The Discrete-time Convolution Sum

For a continuous system
C o n tin u o u s
f(t) s y s te m y (t)
( h (t), H (s ))
the output y(t), in response to an input f (t), is given by the convolution integral:

y(t) = f ( )h(t )d
0
where h(t) is the system impulse response.
For a LTI discrete-time system, such as dened by a dierence equation, we dene the
pulse response sequence {h(n)} as the response to a unit-pulse input sequence {n }, where

1 n=0
n =
0 otherwise.
{d n } {h n }
D S P y
fn a lg o r ith m n
.... -2 -1 0 1 2 .... n .... -2 -1 0 1 2 .... n
If the input sequence {fn } is written as a sum of weighted and shifted pulses, that is

fn = fk nk
k=
then by superposition the output will be a sequence of similarly weighted and shifted pulse
responses

yn = fk hnk
k=
which denes the convolution sum, which is analogous to the convolution integral of the
continuous system.
3 The z-Transform
The z-transform in discrete-time system analysis and design serves the same role as the
Laplace transform in continuous systems. We begin here with a parallel development of
both the z and Laplace transforms from the Fourier transforms.
133
The Laplace Transform The Z transform
(1) We begin with causal f (t) and nd its (1) We sample f(t) at intervals T to produce
Fourier transform (Note that because f (t) is f (t). We take its Fourier transform (and use
causal, the integral has limits of 0 and ): the sifting property of (t)) to produce

F (j) = f (t)ejt dt
F (j) = fn ejnT
0
n=0
(2) We note that for some functions f (t) (for (2) We note that for some sequences fn (for
example the unit step function), the Fourier example the unit step sequence), the summa
integral does not converge. tion does not converge.
(3) We introduce a weighted function (3) We introduce a weighted sequence

w(t) = f (t)et {wn } = fn rn
and note and note

lim w(t) = f (t) lim {wn } = {fn }
0 r1
The eect of the exponential weighting by et The eect of the exponential weighting by rn
is to allow convergence of the integral for a is to allow convergence of the summation for
much broader range of functions f (t). a much broader range of sequences fn .
(4) We take the Fourier transform of w(t) (4) We take the Fourier transform of wn

n jnT
W (j) = F (j|) = f (t)et ejt dt W (j) = F (j|r) = fn r e
0 n=0
= f (t)e(+j) dt
n
0 = fn rejT
n=0
and dene the complex variable s = + j so and dene the complex variable z = rejT so
that we can write that we can write

F (s) = F (j|) = f (t)est dt F (z) = F (j|r) = fn z n
0
n=0
F (s) is the one-sided Laplace Transform. Note F (z) is the one-sided Z-transform. Note that
that the Laplace variable s = + j is ex z = rejT is expressed in polar form.
pressed in Cartesian form.
134
The Laplace Transform (contd.) The Z transform (contd.)

(5) For a causal function f (t), the region of (5) For a right-sided (causal) sequence {fn }
convergence (ROC) includes the s-plane to the the region of convergence (ROC) includes the
right of all poles of F (j). z-plane at a radius greater than all of the poles
of F (z).
jW
{ z }
s - p la n e R O C
z - p la n e
R O C
: : s
: : { z }
-1 .0 1 .0
u n it c ir c le
(6) If the ROC includes the imaginary axis, (6) If the ROC includes the unit circle, the
the FT of f (t) is F (j): DFT of {fn }, n = 0, 1, . . . , N 1. is {Fm }
where
F (j) = F (s) |s=j
Fm = F (z) |z=ejm = F (ejm ),
where m = 2m/N for m = 0, 1, . . . , N 1.

(7) The convolution theorem states (7) The convolution theorem states

L

f (t)g(t) = f ( )g(t )d F (s)G(s) {fn } {gn } =
Z
fm gnm F (z)G(z)

m=
(8) For an LTI system with transfer function (8) For a discrete LSI system with transfer
H(s), the frequency response is function H(z), the frequency response is
H(s) |s=j = H(j) H(z) |z=ej = H(ej ) ||
if the ROC includes the imaginary axis. if the ROC includes the unit circle.
From the above derivation, the Z-transform of a sequence {fn } is

F (z) = fn z n
n=
where z = r ej is a complex variable. For a causal sequence fn = 0 for n < 0, the transform
135
can be written

F (z) = fn z n
n=0
Example: The nite sequence {f0 , . . . , f3 } = {5, 3, 1, 4} has the z-transform

F (z) = 5z 0 + 3z 1 z 2 + 4z 3
The Region of Convergence: For a given sequence, the region of the z-plane in which
the sum converges is dened as the region of convergence (ROC). In general, within the ROC

n
fn r <
n=
and the ROC is in general an annular region of the z-plane:

(z )
z - p la n e r1
r2
(z )
R O C
F (z ) c o n v e rg e s fo r
r1 < r < r2
(a) The ROC is a ring or disk in the z-plane.

(b) The ROC cannot contain any poles of F (z).
(c) For a nite sequence, the ROC is the entire z-plane (with the possible exception of z = 0
and z = .
(d) For a causal sequence, the ROC extends outward from the outermost pole.
(e) for a left-sided sequence, the ROC is a disk, with radius dened by the innermost pole.
(f ) For a two sided sequence the ROC is a disk bounded by two poles, but not containing
any poles.
(g) The ROC is a connected region.
z-Transform Examples: In the following examples {un } is the unit step sequence,

0 n<0
un =
1 n0
and is used to force a causal sequence.
136
(1) {fn } = {n } (the digital pulse sequence)
From the denition of F (z):
F (z) = 1z 0 = 1 for all z.
(2) {fn } = {an un }

n n
n
F (z) = a z = az 1
n=0 n=0
Z 1 z
{an } F (z) = 1
= for |z | > a.
1 az za
since

1
xn = for x < 1.
n=0
1x
(3) {fn } = {un } (the unit step sequence).

1 z
F (z) = z n = 1
= for |z | < 1
n=0
1z z1
from (2) with a = 1.

(4) {fn } = ebn un .

bn n
n
F (z) = e z = eb z 1
n=0 n=0
Z 1 z
ebn F (z) = b 1
= for |z | > eb .
1 e z z ebn
from (2) with a = eb .

(5) {fn } = eb|n
| .
0

n n
F (z) = eb z + eb z 1 1
n= n=0
1 1
= + 1
1 e z 1 eb z 1
b
Note that the item f0 = 1 appears in each sum, therefore it is necessary to subtract 1.
Z 1 e2b
eb|n | F (z) = for eb < |z | < eb .
(1 eb z)(1 eb z 1 )
137
(z )
z - p la n e
e -b e b
X X (z )
R O C
(6) {fn } = { ej0 n un } = {cos(0 n)un } j {sin(0 n)un } .

F (z) = Z {cos(0 n)un } jZ {sin(0 n)un }
From (1)
F (z) = for |z | > 1

1 ej0 z 1
1 cos(0 )z 1 j sin(0 )
=
1 2 cos(0 )z 1 + z 2
z 2 cos(0 )z j sin(0 )z 2
=
z 2 2 cos(0 )z + 1
and therefore
z 2 cos(0 )z
Z {cos(0 n)un } = for |z | > 1
z 2 2 cos(0 )z + 1
sin(0 )z 2
Z {sin(0 n)un } = 2 for |z | > 1
z 2 cos(0 )z + 1
Properties of the z-Transform: Refer to the texts for a full description. We simply
summarize some of the more important properties here.
(a) Linearity:
Z
a {fn } + b {gn } aF (z) + bG(z) ROC: Intersection of ROCf and ROCg .
(b) Time Shift:
Z
{fnm } z m F (z) ROC: ROCf except for z = 0 if k < 0, or z = if k > 0.
If gn = fnm ,

n
G(z) = fnm z = fk z (k+m) = z m F (z).
n= k=
This is an important property in the analysis and design of discrete-time systems. We

will often have recourse to a unit-delay block:
138
fn U n it D e la y y = fn
n -1
z -1
(c) Convolution:
Z
{fn } {gn } F (z)G(z) ROC: Intersection of ROCf and ROCg .

where {fn } {gn } = fk gnk is the convolution sum.
k=
Let

Y (z) = yn z n = fk gnk z n
n= n= k=

(nk) k k
= fk gnk z z = fk z gm z m
k= n= k= m=
= F (z)G(z)
(d) Conjugation of a complex sequence:

Z
f n F (z) ROC: ROCf
(e) Time reversal:
Z 1 1
{fn } F (1/z) ROC: < |z | <
r1 r2
where the ROC of F (z) lies between r1 and r2 .
(e) Scaling in the z-domain:
Z
{an fn } F (a1 z) ROC: |a| r1 < |z| < |a| r2
(e) Dierentiation in the z-domain:
Z dF (z)
{nfn } z ROC: r2 < |z | < r1
dz
139
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 141
Reading:
Proakis & Manolakis, Chapter 3 (The z-transform)
Oppenheim, Schafer & Buck, Chapter 3 (The z-transform)
1 The Discrete-Time Transfer Function

Consider the discrete-time LTI system, characterized by its pulse response {hn }:
c o n v o lu tio n
{fn } {y } = { f h }
L T I s y s t e m
n n n
Z Z
h n
F (z ) Y (z ) = F (z )H (z )
We saw in Lec. 13 that the output to an input sequence {fn } is given by the convolution
sum:

yn = fn hn = fk hnk = hk fnk ,
k= k=
where {hn } is the pulse response. Using the convolution property of the z-transform we have
at the output
Y (z) = F (z)H(z)
where F (z) = Z {fn }, and H(z) = Z {hn }. Then
Y (z)
H(z) =
F (z)
is the discrete-time transfer function, and serves the same role in the design and analysis
of discrete-time systems as the Laplace based transfer function H(s) does in continuous
systems.
1 c D.Rowell 2008
copyright
141
In general, for LTI systems the transfer function will be a rational function of z, and may
be written in terms of z or z 1 , for example
N (s) b0 + b1 z 1 + b2 z 2 + . . . + bM z M
H(z) = =
D(s) a0 + a1 z 1 + a2 z 2 + . . . + aN z N
where the bi , i = 0, . . . , m, ai , i = 0, . . . , n are constant coecients.
2 The Transfer Function and the Dierence Equation

As dened above, let
Y (z) b0 + b1 z 1 + b2 z 2 + . . . + bM z M
H(z) = =
F (z) a0 + a1 z 1 + a2 z 2 + . . . + aN z N
and rewrite as

a0 + a1 z 1 + a2 z 2 + . . . + aN z N Y (z) = b0 + b1 z 1 + b2 z 2 + . . . + bM z M F (z)
If we apply the z-transform time-shift property
Z {fnk } = z k F (z)
term-by-term on both sides of the equation, (eectively taking the inverse z-transform)
a0 yn + a1 yn1 + a2 yn2 + . . . + aN ynN = b0 fn + b1 fn1 + b2 fn2 + . . . + bM fnM
and solve for yn

1 1
yn = (a1 yn1 + a2 yn2 + . . . + aN ynN ) + (b0 fn + b1 fn1 + b2 fn2 + . . . + bM fnM )
a0 a0
ai
N bi
M
= yni + fni
i=1
a0 i=0
a0
which is in the form of a recursive linear dierence equation as discussed in Lecture 13.
The transfer function H(z) directly denes the computational dif

ference equation used to implement a LTI system.
Example 1
Find the dierence equation to implement a causal LTI system with a transfer
function
(1 2z 1 )(1 4z 1 )
H(z) =
z(1 12 z 1 )
Solution:
z 1 6z 2 + 8z 3
H(z) =
1 12 z 1
142
from which
1
yn yn1 = fn1 6fn2 + 8fn3 ,
2
or
1
yn = yn1 + (fn1 6fn2 + 8fn3 ).
2
The reverse holds as well: if we are given the dierence equation, we can dene the system
transfer function.
Example 2
Find the transfer function (expressed in powers of z) for the dierence equation
yn = 0.25yn2 + 3fn 3fn1
and plot the system poles and zeros on the z-plane.

Solution: Taking the z-transform of both sides
Y (z) = 0.25z 2 Y (z) + 3F (z) 3z 1 F (z)
and reorganizing
Y (z) 3(1 z 1 ) 3z(z 1)

H(z) = = = 2
F (z) 1 0.25z 2 z 0.25
which has zeros at z = 0, 1 and poles at z = 0.5, 0.5:
{z }
z - p la n e
x o x o {z }
-0 .5 0 .5 1
143
3 Introduction to z-plane Stability Criteria

The stability of continuous time systems is governed by pole locations - for a system to be
BIBO stable all poles must lie in the l.h. s-plane. Here we do a preliminary investigation of
stability of discrete-time systems, based on z-plane pole locations of H(z).
Consider the pulse response hn of the causal system with
z 1
H(z) = =
za 1 az 1
with a single real pole at z = a and with a dierence equation
yn = ayn1 + fn .
{z }
z - p la n e
a < -1 -1 < a < 0 0 < a < 1 a > 1

o x {z }
1
p o le lo c a tio n
Clearly the pulse response is

1 n=0
hn =
an n1
The nature of the pulse response will depend on the pole location:
0 < a < 1: In this case hn = an will be a decreasing function of n and limn hn = 0 and
the system is stable.
a = 1: The dierence equation is yn = yn1 + fn (the system is a summer and the impulse
response is hn = 1, (non-decaying). The system is marginally stable.
a > 1: In this case hn = an will be a increasing function of n and limn hn = and the
system is unstable.
1 < a < 0: In this case hn = an will be a oscillating but decreasing function of n and
limn hn = 0 and the system is stable.
a = 1: The dierence equation is yn = yn1 + fn and the impulse response is hn = (1)n ,
that is a pure oscillator. The system is marginally stable.
a < 1: In this case hn = an will be a oscillating but increasing function of n and limn |hn | =
and the system is unstable.
144
This simple demonstration shows that this system is stable only for the pole position 1 <
a < 1. In general for a system
M
k=1 (z zk )
H(z) = K N
k=1 (z pk )
having complex conjugate poles (pk ) and zeros (zk ) :
A discrete-time system will be stable only if all of the poles of its
transfer function H(z) lie within the unit circle on the z-plane.
4 The Frequency Response of Discrete-Time Systems

Consider the response of the system H(z) to an innite complex exponential sequence
fn = A ej n = A cos(n) + jA sin(n),
where is the normalized frequency (rad/sample). The response will be given by the con
volution

yn = hk fnk = hk A ej (nk)
k= k=

j k
= A hk e ej n
k=
j j n
= AH(e )e
where the frequency response function H(ej ) is
H(ej ) = H(z)|z=ej
that is
The frequency response function of a LTI discrete-time system is

H(z) evaluated on the unit circle - provided the ROC includes the
unit circle. For a stable causal system this means there are no poles
lying on the unit circle.
{z }
0 < w < p z - p la n e
j1
w i n c r e a s i n g e jw
w = 0
w = p w
{z }
w = -p -1 1
- j1
- p < w < 0
145
Alternatively, the frequency response may be based on a physical frequency associated

with an implied sampling interval T , and
H(ej T ) = H(z)|z=ej T
which is again evaluated on the unit circle, but at angle T .
{z }
0 < W < p /D T z - p la n e
j1
W in c r e a s in g e jW D T
N y q u is t fr e q u e n c y
W = 0
W = p /D T W D T
{z }
W = -p /D T -1 1
- j1
- p /D T < W < 0
From the denition of the DTFT based on a sampling interval T

H (j) = hn emjnT = H(z)|z= emjnT
n=0
we can dene the mapping between the imaginary axis in the s-plane and the unit-circle in
the z-plane
s = j o z = ej o T
jW {z }
N y q u is t fr e q u e n c y s - p la n e m a p p in g
z - p la n e
jp / D T j1
jW o D T
e
jW o
W = 0
" p r im a r y " s tr ip W = p /D T W D T
{z }
-1
s
W = -p /D T 1
- j1
- jp / D T
The periodicity in H( ej T ) can be clearly seen, with the primary strip in the s-plane
(dened by /T < < /T ) mapping to the complete unit-circle. Within the primary
strip, the l.h. s-plane maps to the interior of the unit circle in the z-plane, while the r.h.
s-plane maps to the exterior of the unit-circle.
146
Aside: We use the argument to dierentiate between the various classes of transfer
functions:
H(s) H(j) H(z) H( ej )

Continuous Continuous Discrete Discrete
Transfer Frequency Transfer Frequency
Function Response Function Response
5 The Inverse z-Transform

The formal denition of the inverse z-transform is as a contour integral in the z-plane,

1
F (z)z n1 dz
2j
where the path is a ccw contour enclosing all of the poles of F (z).
Cauchys residue theorem states

1
F (z) dz = Res [F (z), pk ]
2j k
where F (z) has N distinct poles pk , k = 1, . . . , N and ccw path lies in the
ROC.
For a simple pole at z = zo
Res [F (z), zo ] = lim (z zo )F (z),

zzo
and for a pole of multiplicity m at z = zo
1 dm1
Res [F (z), zo ] = lim (z zo )m F (z)
zzo (m 1)! dz m1
The inverse z-transform of F (z) is therefore

fn = Z 1 {F (z)} = Res F (z)z n1 , pk .
k
Example 3
A rst-order low-pass lter is implemented with the dierence equation
yn = 0.8yn1 + 0.2fn .
Find the response of this lter to the unit-step sequence {un }.
147
Solution: The lter has a transfer function
Y (z) 0.2 0.2z
H(z) = = =
F (z) 1 0.8z 1 z 0.8
The input {fn } = {un } has a z-transform

z
F (z) =
z1
so that the z-transform of the output is
0.2z 2
Y (z) = H(z)U (z) =
(z 1)(z 0.8)
and from the Cauchy residue theorem

yn = Res Y (z)z n1 , 1 + Res Y (z)z n1 , 0.8
= lim(z 1)Y (z)z n1 + lim (z 0.8)Y (z)z n1
z1 z0.8
n+1 n+1
0.2z 0.2z
= lim + lim
z 0.8
z1 z0.8 z1
= 1 0.8n+1
which is shown below

y n
0 .8
0 .6
0 .4
0 .2
0
0 2 4 6 8 1 0 1 2 1 4 1 6 1 8
n ( s a m p le s )
Example 4
Find the impulse response of the system with transfer function
1 z2 z2
H(z) = = =
1 + z 2 z2 + 1 (z + j 1)(z j 1)
148
Solution: The system has a pair of imaginary poles at z = j 1. From the

Cauchy residue theorem

hn = Z 1 {H(z)} = Res H(z)z n1 , j 1 + Res H(z)z n1 , j 1
z n+1 z n+1
= lim + lim
zj1 z + j 1 zj 1 z j 1
1 1
= (j 1)n+1 (j 1)n+1
j2 j2
n
j
= 1 + (1)n+1
2
0 n odd
hn = n/2
(1) n even
= cos(n/2)
where we note that the system is a pure oscillator (poles on the unit circle) with
a frequency of half the Nyquist frequency.
y n
1
0 .5
2 6 1 0 1 4 1 8
0
4 8 1 2 1 6 n
-0 .5
-1
Example 5
Find the impulse response of the system with transfer function
1 z2 z2
H(z) = = =
1 + 2z + z 2 z 2 + 2z + 1 (z + 1)2
Solution: The system has a pair of coincident poles at z = 1. The residue at
z = 1 must be computed using
1 dm1
Res [F (z), zo ] = lim (z zo )m F (z).
zzo (m 1)! dz m1
With m = 2, at z = 1,
1 d
Res H(z)z n1 , 1 = lim (z 1)2 H(z)z n1
z1 (1)! dz
d n+1
= lim z
z1 dz
= (n + 1)(1)n
149
The impulse response is

hn = Z 1 {H(z)} = Res H(z)z n1 , 1 = (n + 1)(1)n .
h n
2 0
1 0
0
2 4 6 8 1 0 1 2 1 4 1 6 1 8 n
-1 0
-2 0
Other methods of determining the inverse z-transform include:
Partial Fraction Expansion: This is a table look-up method, similar to the method
used for the inverse Laplace transform. Let F (z) be written as a rational function of
z 1 :
M k
k=0 bi z
F (z) = N
k
k=0 ai z
M 1
k=1 (1 ci z )
= N
k=1 (1 di z )
1
If there are no repeated poles, F (z) may be expressed as a set of partial fractions.

N
Ak
F (z) =
k=1
1 dk z 1
where the Ak are given by the residues at the poles
Ak = lim (1 dk z 1 )F (z).
zdk
Since
Z 1
(dk )n un
1 dk z 1

N
fn = Ak (dk )n un .
k=1
1410
Example 6
Find the response of the low-pass lter in Ex. 3 to an input
fn = (0.5)n
Solution: From Ex. 3, and from the z-transform of {fn },

1 0.2
F (z) = , H(z) =
1 0.5z 1 1 0.8z 1
so that
0.2
Y (z) =
(1 + 0.8z 1 )
0.5z 1 )(1
A1 A2
= +
1 + 0.5z 1 1 0.8z 1
Using residues
0.2 0.1
A1 = lim =
1 0.8z
z0.5 1 1.3
0.2 0.16
A2 = lim =
z0.8 1 + 0.5z 1 1.3
and

0.1 1 1 0.16 1 1
yn = Z + Z
1.3 1 + 0.5z 1 1.3 1 0.8z 1
0.1 0.16
= (0.5)n + (0.8)n
1.3 1.3
Note: (1) If F (z) contains repeated poles, the partial fraction method must be ex
tended as in the inverse Laplace transform.
(2) For complex conjugate poles combine into second-order terms.
Power Series Expansion: Since

F (z) = fn z n
n=
if F (z) can be expressed as a power series in z 1 , the coecients must be fn .
1411
Example 7
Find Z 1 {log(1 + az 1 )}.
Solution: F (z) is recognized as having a power series expansion:

(1)n+1 an
1
F (z) = log(1 + az ) = z n for |a| < |z|
n=1
n
Because the ROC denes a causal sequence, the samples fn are

0 for n 0
fn = (1)n+1 an
for n 1.
n
Polynomial Long Division: For a causal system, with a transfer function written as
a rational function, the rst few terms in the sequence may sometimes be computed
directly using polynomial division. If F (z) is written as
N (z 1 )
F (z) = = f0 + f1 z 1 + f2 z 2 + f2 z 2 +
D(z 1 )
the quotient is a power series in z 1 and the coecients are the sample values.
Example 8
Determine the rst few terms of fn for
1 + 2z 1
F (z) =
1 2z 1 + z 2
using polynomial long division.
Solution:
1 + 4z 1 + 7z 2 +

1 2z 1 + z 2 1 + 2z 1
1 2z 1 + z 2
4z 1 z 2
4z 1 8z 2 + 4z 3
7z 2 4z 3
so that
1 + 2z 1
F (z) = = 1 + 4z 1 + 7z 2 +
1 2z 1 + z 2
and in this case the general term is
fn = 3n + 1 for n 0.
1412
In general, the computation can become tedious, and it may be dicult to recognize
the general term from the rst few terms in the sequence.
1413
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 151
Reading:
Proakis & Manolakis, Ch. 7
Oppenheim, Schafer & Buck. Ch. 10
Cartinhour, Chs. 6 & 9
1 Frequency Response and Poles and Zeros

As we did for the continuous case, factor the discrete-time transfer functions into as set of
poles and zeros;
M
b0 z 0 + b1 z 2 + + bM z M i=1 (z zi )
H(z) = 0 2 N
= K N
a0 z + a1 z + + aN z i=1 (z pi )
where Zi are the system zeros, the pi are the system poles, and K = b0 /a0 is the overall gain.
We note, as in the continuous case that the polse and zeros must be either real, or appear
in complex conjugate pairs.
As in the continuous case, we can draw a set of vectors from the poles and zeros to a test
point in the z-plane, and evaluate H(z) in terms of the lengths and angles of these vectors.
In particular, we choose to evaluate H( ej ) on the unit circle,
M j
j i=1 e zi
H( e ) = K N
j p )
i=1 ( e i
and
M j M
e zi
H( ej ) = K i=1 i=1 qi
N
= K N
j p |
i=1 | e i i=1 ri
M
N M N
j
H( ej ) = j
e zi e pi = i i
i=1 i=1 i=1 i=1
where the qi and i are the lengths and angles of the vectors from the zeros to the point
z = ej , the ri and i are the lengths and angles of the vectors from the poles to the point
z = ej , as shown below:
1 c D.Rowell 2008
copyright
151
{z }
j1 A t fre q u e n c y w :
z = e jw
|H ( e jw ) | = K
p r1 q 1 q 2
X
1
f 1
r1 r2 r3
q 1 r3
q
2
H ( e jw ) = ( q 1
+ q 2 )
r2
- (f 1 + f 2 + f 3
)
q f q
O X O {z }
1 3 2
-1 z 1 p z 1
3 2
f
X 2
p 2
- j1
We can interpret the eect of pole and zero locations on the frequency response as follows:

(a) A pole (or conjugate pole pair) on the unit circle will cause H( ej ) to become innite
at frequency .

(b) A zero (or conjugate zero pair) on the unit circle will cause H( ej ) to become zero at
frequency .

(c) Poles near the unit circle will cause a peak in H( ej ) in the neighborhood of those
poles.

(c) Zeros near the unit circle will cause a dip, or notch, in H( ej ) in the neighborhood of
those zeros.

(d) Poles and zeros at the origin z = 0 have no eect upon H( ej ), but add a frequency
dependent linear phase taper ( for a pole, + for a zero), which is equivalent to
shift.

(e) A pole or zero at z = 1 forces H( ej ) to be innite (pole) or 0 (zero) at = 0.

(f ) A pole or zero at z = 1 forces H( ej ) to be innite (pole) or 0 (zero) at = , (the
Nyquist frequency).
2 FIR Low-Pass Filter Design

The FIR (nite impulse response) lter is an all-zero system with a dierence equation
M

yn = bk fnk
k=0
which is clearly a convolution of the input sequence with an impulse response {hk } = {bk },
for k = 0, , M . A direct-form causal implementation is
152
fn fn -1 f n -2 fn -3 fn -M
-1 -1 -1 -1
z z z z
b 0 b 1 b 2
b 3 b M
+ + + +
y
+ + + + n

M
M
1
H(z) = bk z k = bk z M k
k=0
z M k=0
and the frequency response is

M

j
H( e ) = bk ej k .
k=0
Note that there are M + 1 terms in the impulse response but the order of the polynomials
is M .
Example 1
Find the frequency response H( ej ) for a simple three-point moving average
lter:
1
yn = (fn + fn1 + fn2 ) .
3
Solution:
1 1 1
H(z) = z 0 + z 1 + z 2
3 3 3
so that
1
H( ej ) = 1 + ej + e2j
3
1 j j
= e e + 1 + ej
3
1
= (1 + 2 cos()) ej
3
and

H( ej ) = 1 (1 + 2 cos())
3
H( ej ) = .
153
jw
|H (e )|
w
- p 0 p
The ideal FIR low-pass lter has a response

j 1 || c
H( e ) =
0 c < ||
{z }
jw jw
|H (e )| H (e ) = 0
jw
H (e ) = 1
1
w c {z }
- w c
- w c w w
- p c p
The impulse response hn = Z 1 {H(z)}, and although we are not given H(z) explicitly, we
can use the formal denition of the inverse z-transform (Lecture 14) as a contour integral in
the z-plane,
1 1
Z {H(z)} = H(z)z n1 dz
2j
where the path is a ccw contour enclosing all of the poles of H(z), and for a stable lter
choose the contour as the unit-circle. Let z = ej , so that dz = j ej d, and
c
1 1 j n c sin(c n)
hn = Z {H(z)} = 1. e d =
2 c c n
The impulse response of the FIR ideal low-pass lter is therefore

c sin(c n)
hn =
c n
154
The following gure shows the central region of the impulse response of an ideal FIR lter
with c = 0.2:
h n
0 .2
0 n
It is obvious that this impulse response has two problems:
(a) It is innite in extent, and
(b) It is non-causal.
To produce a causal, nite length lter

(a) Truncate {hn } to include M +1 central points (M +1 odd), that is select the points
M/2 n M/2. Let this truncated lter be designated H (z).
(b) Shift the truncated impulse response {h n } to the right by M/2 to form a causal

sequence {hn }, where hn = h(M/2n) , for n = 0, . . . M .
Take {hn } as the FIR causal approximation to the ideal low-pass lter.
Then
H (z) = z (M 1)/2 H
(z),
that is the response is delayed by (M 1)/2 samples. The frequency response is
H ( ej ) = ej (M 1)/2 H
( ej ),
( ej ) is real
and because H
j
H ( e ) = H
( ej )
H ( ej ) = (M 1)/2.
155
Example 2
Design a ve point causal FIR low-pass lter with a cut-o frequency c = 0.4.
Solution: The ideal lter has an impulse response

c sin(c n) 1 sin(n/2)
hn = =
c n 2 n/2
Select M + 1 = 5, and select the ve central components:
n: 2 1 0 1 2
hn : 0.0935 0.3027 0.4 0.3027 0.0935
n }:
Shift to the right by M/2 = 2, and form the causal impulse response {h
n: 0 1 2 3 4
n : 0.0935 0.3027 0.4 0.3027 0.0935
h
with dierence equation
yn = 0.0935fn + 0.3027fn1 + 0.4fn2 + 0.3027fn3 + 0.0935fn4
The causal impulse response and the frequency response magnitude are shown
below:
h | H ( e jw ) |
n
1 .2
0 .4
0 .3 0 2 7
0 .8
0 .6
0 .4
0 .0 9 3 5
0 .2
n w
-4 -2 0 2 4 6 8 0 0 .5 1 1 .5 2 2 .5 3
2.1 The Eect of Truncation and Shifting

(a) The Eect of Truncation The selection of the M + 1 central components of the
non-causal impulse response {hn } can be written as a product
{hn } = {hn rn }
where {rn } is an even rectangular window function

1 |n| M/2
rn =
0 otherwise.
The following gure shows {rn } for M = 20.
156
n
1
n
-1 0 0 1 0
The truncated frequency response is therefore

1
H ( ej ) = H( ej ) R( ej ).
2
For the window function
M/2

R(z) = z 1
k=M/2
M/2 M/2

j j k
R( e ) = e =1+ 2 cos(k) = DM/2 ()
k=M/2 k=1
DM/2 () is known as the Dirichlet kernel, and is found in the study of truncated Fourier
series and convolution of periodic functions. It is easy to show (using the sum of a
nite geometric series) that
sin((M + 1)/2)
R( ej ) = DM/2 () =
sin(/2)
R ( e jw )
M + 1
1
M = 1 0
M = 2 0
M = 3 0
w
- p 0 p
157
Notice (1) The width of the main lobe decreases with M , (2) the side lobes do not
decay to zero as increases.
Aside: The formal denition of the z-transform of the product of two sequences is
given by the z-plane contour integral

z
1
F (z) = Z {xn yn } = X()Y 1 d
2j
where X(z) = Z {xn }, Y (z) = Z {yn }, and the contour lies in the ROC of both X(z)
and Y (z). In particular if the unit-circle lies within the ROC of both sequences, choose
the unit-circle as the contour, = ej , then

j o 1
F(e ) = X( ej )Y ( ej (0 ) ) d
2
which is the convolution of X( ej ) and Y ( ej ).
The frequency response of the truncated lter is

1
H ( ej ) = H( ej ) R( ej )
2

1
= H( ej )R( ej () ) d
2
or c
1 j
H (e ) = DM/2 ( ) d.
2 c
which is shown below for lter with c = 0.4 and lengths M + 1 = 11, 21, 31, and 41:
| H '( e jw
)|
M = 3 0
1
M = 4 0
M = 1 0
M = 2 0
0 .0 9 1
0 w
0 .4 p p
158
In general:
The amplitude of the ripple in the pass-band does not decrease with the lter
order M .
Similarly, the stop-band attenuation is relatively unaected by the lter order,
and the amplitude of the rst side-lobe is 0.091, so that the truncated lter has
a stop-band attenuation of -21 dB.
The width of the transition-band decreases with increasing M .
(b) The Eect of the Right-Shift to Form a Causal Filter The truncated non-causal
impulse response {h } is even and real, so that H
( ej ) is also real and even, that is

H ( ej ) = 0.
The right-shift of h by M/2 samples to force causality imposes
(z) = z M/2 H (z)
and therefore
( ej ) = ej M/2 H ( ej ).
H
The phase response of the lter is
H ( ej ) = (M/2)
which is a linear phase taper (lag).
The eect of the right-shift on the impulse response of the ideal lter is to impose
a phase lag that is proportional to frequency, with a slope of M/2.
159
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 161
Reading:
Proakis & Manolakis, Sec. 10.2
Oppenheim, Schafer & Buck, Chap. 7.
Cartinhour, Chap. 9.
1 FIR Low-Pass Filter Design by Windowing

In Lecture 15 we examined the creation of a causal FIR lter based upon an ideal low-pass
lter with cut-o frequency c , and found that the impulse response was

c sin(c n
h(n) = < n < .
c n
The resulting lter is therefore both innite in extent and non-causal.

To create a nite length lter we truncated the impulse response by multiplying {h(n)}
with an even rectangular window function {r(n)} of length M + 1, where

1 |n| M/2
r(n) =
0 otherwise.
The result was to create a modied lter {hn } with a real frequency response function
H ( ej ) from the convolution

j 1
H (e ) = H( ej )R( ej () ) d
2
where
sin((M + 1)/2)
R( ej ) =
sin(/2)
The truncation generates a Gibbs phenomenon associated with the band edges of H ( ej )
where, as demonstrated in the gure below:
(a) Both the pass-band and the stop-band exhibit signicant ripple, and the maxima of the
ripple is relatively independent of the chosen lter length M + 1.
1 c D.Rowell 2008
copyright
161
(b) The amplitude of the rst side-lobe in the stop-band ia approximately 0.091, corre
sponding to an attenuation of 21 dB, at that frequency.
(c) The width of the transition region decreases with M + 1, the lter length.
| H '( e jw
)|
M = 3 0
1
M = 4 0
M = 1 0
M = 2 0
0 .0 9 1
0 w
0 . 4 p p
A causal lter was then formed by applying a right-shift of M/2 to the impulse response to
form {h n } where
(n) = h (n M/2)
h 0 n M + 1.

The shift was seen to have no eect on H( ej ), but created a linear phase taper (lag).
The windowing method of FIR seeks to improve the lter characteristic by selecting
an alternate length M + 1 window function {w(n)} with improved spectral characteristics

j j
W ( e ), which when convolved with the ideal low-pass lter function H( e ) will produce
a better lter.
There are many window functions available. We rst look at three common xed param
eter windows:
The Bartlett Window: The length M + 1 Bartlett window is a even triangular window

1
+ 2n/M M/2 n 0
w(n) = 1 2n/M 0 n M/2

0 otherwise,

as shown for M + 1 = 40 in the gure below. Also plotted is the spectrum ( e j ,
W
and for comparison the spectrum of the same length rectangular window R( ej .
It can be seen that the main lobe of the Bartlett spectrum is wider than that of the rect
angular window, but that the side-lobes decrease in amplitude much faster at higher
frequencies. The Bartlett window produces a monotonically decreasing frequency re
sponse magnitude, as is shown below.
162
w n 2 0 lo g ( | W ( e jw ) / W ( 1 ) | )
1 0
1 0 w
p
M = 4 0
B a r tle tt
N o r m a liz e d M a g n itu d e ( d B )
-2 0
R e c ta n g u la r
-4 0
-6 0
n -8 0
-2 0 -1 0 0 1 0 2 0
The Hann (or Hanning) Window: The Hann window is a raised cosine window

0.5 + 0.5 cos 2M n M/2 n M/2
w(n) =
0 otherwise.
The Hann window, along with its spectrum, is shown for M + 1 = 40 below. As
with the Bartlett example above, the spectrum of the rectangular window is given for
comparison. Again it can be seen that the Hann window has a broader main lobe, but
with much reduced side-lobes (even compared to the Bartlett window) away from the
main peak.
w n 2 0 lo g ( | W ( e jw ) / W ( 1 ) | )
1 1 0
0 w
= 4 0 p
M
H a n n
-2 0 R e c ta n g u la r
-4 0
-6 0
n -8 0
-2 0 -1 0 0 1 0 2 0
The Hamming Window: The Hamming window is another raised cosine window, but
this time on a pedestal.

0.54 + 0.46 cos 2M n M/2 n M/2
w(n) =
0 otherwise.
so that at the extremities (n = M/2), the value wM/2 = 0.08. From the gure below,
it can be seen that the Hamming widow has smaller side-lobes close to the main lobe,
but that the side-lobes distant from the main peak have a higher amplitude.
163
w 2 0 lo g ( | W ( e jw ) / W ( 1 ) | )
1
n
1 0
= 4 0
0 w
M
p
H a m m in g
-2 0 R e c ta n g u la r
-4 0
-6 0
n -8 0
-2 0 -1 0 0 1 0 2 0
Filter Design Procedure Using a Fixed Window:

The only design parameters available when using a xed window are (1) the low-pass cut-o
frequency c , (2) the choice of window type, and (3) the lter length M + 1. Once these
choices are made, the procedure is as follows
(a) Form the samples of the ideal low-pass lter of length M + 1.

c sin(c n)
h(n) = for M/2 n M/2
c n
(b) Form the length M + 1 window {wn } of the chosen type.
(c) Form the impulse response {hn } where hn = hn wn .
(d) Shift all samples to the right by M/2 samples.
Example 1
Write some MATLAB code to design a length 41 low-pass FIR lter with cut-o
frequency c = 0.4 using a Hamming window. Plot the magnitude and pahase
of the resulting lter.
Solution: The following MATLAB code was used:
n=-20:20;
wc=0.4*pi;
h = (wc/pi)*sinc(wc/pi*n);
hprime = h.*hamming(41);
% All done - no need to shift - just interpret hprime as the shifted
% impulse response.
% Plot the frequency response:
freqz(hprime,1);
which generated the following frequency response plots:
164
0
M a g n itu d e ( d B )
- 2 0
- 4 0
- 6 0
- 8 0
0 0 .1 0 .2 0 .3 0 .4 0 .5 0 .6 0 .7 0 .8 0 .9 1
N o r m a liz e d F r e q u e n c y ( x p r a d /s a m p le )
-5 0 0
P h a s e (d e g re e s )
-1 0 0 0
-1 5 0 0
-2 0 0 0
0 0 .1 0 .2 0 .3 0 .4 0 .5 0 .6 0 .7 0 .8 0 .9 1
N o r m a liz e d F r e q u e n c y ( x p r a d /s a m p le )
Note that the linear phase characteristic has jump discontinuities of (or 180 )
when H ( ej ) changes sign.
The following gure shows a comparison of length 41 lters designed with the Bartlett, Hann,
and Hamming windows.
1 0
0 w c = 0 .4 p r a d /s a m p le
-1 0 M + 1 = 4 0
F r e q u e n c y R e s p o n s e M a g n itu d e ( d B )
-2 0
B a r tle tt w in d o w
-3 0
-4 0
H a m m in g w in d o w
-5 0
H a n n w in d o w
-6 0
-7 0
-8 0
-9 0
-1 0 0
0 0 .5 1 1 .5 2 2 .5 3
F re q u e n c y w
165
Notice that while the Bartlett window generates a lter with less attenuation in the stop-
band, it has no ripple in the stop-band (no sign changes in H ( ej )) and therefore no jump
discontinuities in its linear phase characteristic.
General Comments on Window Taper Consider the family of window functions that
are raised cosine functions on a pedestal, characterized by

+ (1 ) cos 2
M
n M/2 n M/2
w (n) =
0 otherwise.
where the parameter , for 1 0.5, denes the degree of taper. When = 1 we have
the rectangular window with zero taper, when = 0.5 we have the Hann window (maximum
taper) , and the Hamming window corresponds to = 0.54.
w n
R e c ta n g u la r
a = 1
in c r e a s in g ta p e r
a = 0 .5
H a n n
H a m m in g
a = 0 .5 4
n
-M /2 0 M /2
These window functions may written as a linear combination of the rectangular window
wrect (n), and the Hann window wHann (n):
w (n) = 2(1 )wHann (n) + 2( 0.5)wrect (n)
The spectra of these windows W ( ej ) will therefore be a similar combination of the spectra
Wrect ( ej ) of the rectangular window, and WHann ( ej ) of the Hann window.
W ( ej ) = 2(1 )WHann ( ej ) + 2( 0.5)Wrect ( ej )
166
jw
2 0 l o g (|W (e )/W (1 )|)
1 0
0
w
p
- 2 0
a = 1 ( R e c t a n g u l a r )
-4 0
-6 0
in c r e a s in g ta p e r
a = 0 .5 (H a n n )
-8 0
Although we have only discussed raised cosine windows here, in general the degree of taper
aects the convolution kernel as follows:
As the taper increases the width of the main lobe increases, causing the transition
band-width in the lter to increase.
As the taper increases the amplitude of the side-lobes decreases more rapidly away
from the main lobe, with the result that the lter stop-band attenuation is signicantly
increased at high frequencies.
The Kaiser Window: The Kaiser window, dened as

I0 1( M/2n
)
2
wK (n, ) = I0 ()
, M2 n M
2

0, otherwise
where I0 () is the zero-order modied Bessel function of the rst kind, and the parameter
provides a convenient control over the window taper (and the resultant trade-o between
lower side-lobe amplitudes and the width of the main lobe). (Note: Some authors dene the
window in terms of a parameter = 2/M .)
Kaiser windows for = 2, 4, 6 are shown below:
w (n ) 2 0 lo g 1 0 ( | W ( e jw ) / W ( 1 ) | ) w
0
p
b = 2 b = 2
-2 0
b = 4
b = 6
-4 0
b = 4
b = 6 -6 0
-2 0 0 2 0
n -8 0
167
The eect of the parameter on the window taper, and the compromise between the width
of the main lobe and sidelobe amplitude can be easily seen. These three window functions
were used to design low-pass FIR lters with c = 0.4. the frequency response magnitudes
are shown below
2 0 lo g 1 0
(|H (e jw
)|)
0 w
p
b = 4
-2 0 b = 2 b = 6
b = 2
b = 4 b = 6
-4 0
-6 0
-8 0
-1 0 0
The compromise between stop-band attenuation and transition steepness can be clearly seen.
The Kaiser window is very commonly used in FIR lters.
2 Window FIR Filters or Other Filter Types

High-Pass Filter: Given an ideal low-pass lter Hlp ( ej ), a high-pass lter Hhp ( ej ) may
be created:
Hhp ( ej ) = 1 Hlp ( ej )
H lp ( e jw ) H h p ( e jw )
1 1
0 w p w w
c 0 w c p
Then the impulse response is

{hhp (n)} = IDFT {1} IDFT Hlp ( ej )
c sin(c n)
= (n)
c n
After windowing to a length M + 1

c sin(c n)
h(n) = w(n) (n) , |n| M/2.
c n
168
The impulse response is then shifted to the right by M/2 samples to make it causal as before.
Band-Pass Filter: A band-pass lter Hbp ( ej ) may be designed from a pair of low-pass
lters Hlpu ( ej ) and Hlpl ( ej ) with cut-o frequencies cu and cl respectively,
Hbp ( ej ) = Hlpu ( ej ) Hlpl ( ej ).
H lp ( e jw ) H b p ( e jw )
1 1
0 w p w w w p
w
w c l c u 0 c l c u
Then

cu sin(cu n) cl sin(cl n)
hbp (n) = w(n) , |n| M/2.
cu n cl n
Band-stop Filter: A band-stop lter Hbs ( ej ) may be designed from a low-pass lters
Hlp ( ej ) and a high-pass lter Hhp ( ej ) with cut-o frequencies cl and cu respectively,
Hbs ( ej ) = Hlp ( ej ) + Hhp ( ej ).
H b s ( e jw )
0 w w c u p w
c l
Then

cu sin(cu n) cl sin(cl n)
hbs (n) = w(n) + (n) , |n| M/2.
cu n cl n
We show below that a linear phase high-pass or band-stop lter must have a length
M + 1 that is odd.
169
3 The Zeros of a Linear Phase FIR Filter

Consider the transfer function of a FIR system with an even-symmetric impulse response of
length M + 1
M
H(z) = hk z k
k=0
The order of the polynomial is M . Also

M
M
M
1 k M (M k) M
H(z ) = hk z = z hk z =z hM n z n
k=0 k=o n=o
where n = M k. Because {hk } is even-symmetric, hk = hM k , and the polynomials in

H(z) and H(z 1 ) are identical
H(z 1 ) = z (M ) H(z).
This means that if z1 is a zero of H(z), that is H(z1 ) = 0, then also H(1/z1 ) = 0, and
therefore 1/z1 is also a zero of H(z).
If z1 = r ej , then 1/z1 = (1/r) ej and the reciprocal zeros may be drawn on the z-plane
{ z }
j1
O z 1
r
q { z }
-1 -q 1
1 /r
O 1 /z 1
- j1
In addition, zeros are either real or appear in complex conjugate pairs, with the result
A general complex zero will be a member of a group of four zeros that are a quad of
reciprocal conjugates.
A pair complex zeros on the unit circle are their own reciprocals, and so will exist only
as a pair.
A general real zero will be a member of a conjugate pair.
A zero at z = 1 will satisfy its own reciprocal, and therefore may exist on its own.
1610
The gure below shows a quad of zeros associated with a complex zero z1 , a conjugate pair
of zeros on the unit circle associated with z2 , a reciprocal pair associated with the real zero
z3 , and a single zero z4 at z = 1.
{ z }
j1
O 1 /z 1
O z 2
O z 1
z 4 z 3 z { z }
O
3
O O
-1 1
z
O
1
O z 2
O 1 /z 1
- j1
In addition

M
1
M
1
M
k M k
H(z) = hk z = M hk z = M hM n z n
k=0
z k=0 z n=0
where n = M k. But with even symmetry h(M n) = h(n), and since (1)k = (1)k ,
at z = 1
H(1) = (1)M H(1).
If M is odd, H(1) = H(1), thus forcing H(1) = 0, therefore H(z) has a zero at
z = 1 if the lter length M + 1 is even.
Any lter with a nite response magnitude at = cannot have a zero at z = 1.

For an even-symmetric FIR lter this requires that the lter length M + 1 be odd
(or equivalently that the number of zeros be even). Linear phase FIR high-pass and
band-stop lters must have an odd lter length.
Example 2
Draw the pole-zero plot for a length 40 low-pass linear-phase FIR lter with
c = 0.4 using a Kaiser window with = 3.
Solution: The plot below was generated with the MATLAB commands:
>> b=fir1(39,0.4,kaiser(40,3));
>> zplane(b,1)
1611
1
0 .8
0 .6
0 .4
0 .2
Im a g in a r y P a r t 0
3 9
-0 .2
-0 .4
-0 .6
-0 .8
-1
-1 -0 .5 0 0 .5 1
R e a l P a rt
The complex reciprocal conjugate quads in the pass-band, conjugate pairs on the
unit circle in the stop-band, and real axis reciprocals can be clearly seen.
Notice that because M + 1 = 40 is even, there is a zero at z = 1, and that this
lter would not be satisfactory for transformation to a high-pass or band-stop
lter.
1612
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 171
Reading:
Class Handout: Frequency-Sampling Filters.
Proakis and Manolakis: Secs. 10.2.3, 10.2.4
Oppenheim, Schafer and Buck: 7.4
Cartinhour: Ch. 9
1 Frequency-Sampling Filters
In the frequency-sampling lters the parameters that characterize the the lter are the
values of the desired frequency response H(ej ) at a discrete set of equally spaced sampling
frequencies. In particular, let
2
k = k k = 0, . . . , N 1 (1)
N
as shown below for the cases of N even, and N odd. Note that when N is odd there is
no sample at the Nyquist frequency, = . The frequency-sampling method guarantees
that the resulting lter design will meet the given design specication at each of the sample
frequencies.
Im (z ) Im (z )
z - p la n e z - p la n e
w 1 w 1
R e (z )
2 p 2 p
1 0 1 1 R e (z )
-1 1 -1 1
N = 1 0 (e v e n ) N = 1 1 (o d d )
(a ) (b )
1 c D.Rowell 2008
copyright
171
For convenience denote the complete sample set {Hk } as

Hk = H(ejk ) k = 1, . . . , N 1.
For a lter with a real impulse response {hn } we require conjugate symmetry, that is
k
HN k = H
and further, for a lter with a real, even impulse response we require {Hk } to be real and
even, that is
HN k = Hk .
Within these constraints, it is sucient to specify frequency samples for the upper half of
the z-plane, that is for

2 k = 0, . . . , N 21 N odd
k = k N
N k = 0, . . . , 2 N even.
and use the symmetry constraints to determine the other samples.
If we assume that H(ej ) may be recovered from the complete sample set {Hk } by the
cardinal sinc interpolation method, that is

N 1
sin ( 2k/N )
H(ej ) = Hk
k=0
2k/N
then H(ej ) is completely specied by its sample set, and the impulse response, of length
N , may be found directly from the inverse DFT,
{hn } = IDFT {Hk }
where
1
N 1
2kn
hn = Hk ej N n = 0, . . . , N 1
N k=0
As mentioned above, this method guarantees that the resulting FIR lter, represented by
{hn }, will meet the specication H(ej ) = Hk at = k = 2k/N . Between the given
sampling frequencies the response H(ej ) will be described by the cardinal interpolation.
1.1 Linear-Phase Frequency-Sampling Filter

The lter described above is nite, with length N , but is non-causal. To create a causal lter
with a linear phase characteristic we require an impulse response that is real and symmetric
about its mid-point. This can be done by shifting the computed impulse response to the
right by (N 1)/2 samples to form
H (z) = z (N 1)/2 H(z)
but this involves a non-integer shift for even N . Instead, it is more convenient to add the
appropriate phase taper to the frequency domain samples Hk before taking the IDFT. The
non-integer delay then poses no problems:
172
Apply a phase shift of
k(N 1)
k = (2)
N
to each of the samples in the upper half z-plane

jk k = 0, . . . , (N 1)/2 (for n odd)
Hk = Hk e
k = 0, . . . , N/2 (for n even)
Force the lower half plane samples to be complex conjugates.

k = 1, . . . , (N 1)/2 (for n odd)
HN k = Hk
k = 1, . . . , N/2 1 (for n even)
Then the linear-phase impulse response is
{hn } = IDFT {Hk }
1.2 A Simple MATLAB Frequency-Sampling Filter

The following is a MATLAB script of a tutorial frequency-sampling lter
h = firfs(samples)
that takes a vector samples of length N of the desired frequency response in the range
0 (N 1)/N , and returns the linear-phase impulse response {hn } of length 2N 1.
-------------------------------------------------------------------------
% 2.161 Classroom Example - firfs - A simple Frequency-Sampling Linear-Phase FIR
% Filter based on DFT interpolation.
% Usage : h = firfs(samples)
% where samples - is a row vector of M equi-spaced, real values
% of the freq. response magnitude.
% The samples are interpreted as being equally spaced around
% the top half of the unit circle at normalized (in terms of
% the Nyquist frequency f_N) frequencies from
% 0 to 2(M-1)/(2M-1) x f_N,
% or at frequencies 2k/(2N-1)xf_N for k = 0...M-1
% Note: Because the length is odd, the frequency response
% is not specified at f_N.
% h - is the output impulse response of length 2M-1 (odd).
% The filter h is real, and has linear phase, i.e. has symmetric
% coefficients obeying h(k) = h(2M+1-k), k = 1,2,...,M+1.
%------------------------------------------------------------------------
function h = firfs(samples)
% Find the length of the input array...
% The complete sample set on the unit circle will be of length (2N-1)
173
%
N = 2*length(samples) -1;
H_d = zeros(1,N);
%
% We want a causal filter, so the resulting impulse response will be shifted
% (N-1)/2 to the right.
% Move the samples into the upper and lower halves of H_d and add the
% linear phase shift term to each sample.
%
Phi = pi*(N-1)/N;
H_d(1) = samples(1);
for j = 2:N/2-1
Phase = exp(-i*(j-1)*Phi);
H_d(j) = samples(j)*Phase;
H_d(N+2-j) = samples(j)*conj(Phase);
end
%
% Use the inverse DFT to define the impulse response.
%
h = real(ifft(H_d));
The following MATLAB commands were used to generate a lter with 22 frequency samples,
generating a length 43 lter.
h=firfs([1 1 1 1 0.4 0 0 0 0 0.8 2 2 2 2 0.8 0 0 0 0 0 0 0 ]);
freqz(h,1)
The lter has two pass-bands; a low-pass region with a gain of unity, and a band-pass region
with a gain of two. Notice that the band-edges have been specied with transition samples,
this is discussed further below. The above commands produced the following frequency
response for the lter.
0
Magnitude (dB)
20
40
60
80
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Normalized Frequency ( rad/sample)
0
Phase (degrees)
500
1000
1500
2000
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Normalized Frequency ( rad/sample)
174
1.3 The Eect of Band-Edge Transition Samples

One of the advantages of the frequency-sampling lter is that the band-edges may be more
precisely specied than the window method. For example, low-pass lters might be specied
by
h = firfs([1 1 1 1 1 0.4 0 0 0 0 0 0]);
with one transition value of 0.4, or
h = firfs([1 1 1 1 0.7 0.2 0 0 0 0 0 0]);
with a pair of transition specications. The frequency-sampling lter characteristic will pass
through these points, and they can have a signicant eect on the stop-band characteristics
of the lter.
The gure below shows the eect of varying the value of a single transition point in a
lter of length N = 33.
10
t = 0.6
0
t = 0.4
t = 0.2
10
Magnitude (dB)
20
30
40
50
60
70
80
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Frequency f/FN
The values shown are for t = 0.6, 0.4 and 0.2. There is clearly a signicant improvement in
the stop-band attenuation for for the case t = 0.4.
Similarly the following gure compares the best of these single transition values (t = 0.4)
with a the response using two transition points (t1 = 0.59, t2 = 0.11). The lter using two
transition points shows a signicant improvement in the stop-band over the single point case,
at the expense of the transition width.
10
0 t1 = .59, t2 = 0.11
t = 0.4
10
20
Magnitude (dB)
30
40
50
60
70
80
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Frequency f/fN
175
Rabiner et al. (1970) did an extensive linear-programming optimization study to deter
mine the optimum value of band edge transition values, and tabulated the results for even
and odd lters of dierent lengths. The results show that for one transition point topt 0.4,
and for two points topt 0.59, and 0.11.
2 FIR Filter Design Using Optimization

These methods allow much greater exibility in the lter specication. In general they seek
the lter coecients that minimize the error (in some sense) between a desired frequency
response Hd ( ej ) and the achieved frequency response H( ej ). The most common optimiza
tion method is that due to Parks and McClellan (1972) and is widely available in software
lter design packages (including MATLAB).
The Parks-McClellan method allows
Multiple pass- and stop-bands.
Is an equi-ripple design in the pass- and stop-bands, but allows independent weighting
of the ripple in each band.
Allows specication of the band edges.
e q u i- r ip p le tr a n s itio n
1 + d 1
1 -d 1
p a s s - b a n d s to p -b a n d
e q u i- r ip p le
d 2
0 w
w c w s p
For the low-pass lter shown above the specication would be
1 1 < H( ej ) < 1 + 1 in the pass-band 0 < c

2 < H( ej ) < 2 in the stop-band s < .
where the ripple amplitudes 1 and 2 need not be equal. Given these specications we need
to determine, the length of the lter M + 1 and the lter coecients {hn } that meet the
specications in some optimal sense.
If M + 1 is odd, and we assume even symmetry
hM k = hk k = 0 . . . M/2
176
and the frequency response function can be written

M/2
j
H( e ) = h0 + 2 hk cos(k)
k=1

M/2
= ak cos(k)
k=0
Let Hd ( ej ) be the desired frequency response, and dene a weighted error

E( ej ) = W ( ej ) Hd ( ej ) H( ej )
where W ( ej ) is a frequency dependent weighting function, but by convention let W ( ej )

be constant across each of the critical bands, and zero in all transition bands. In particular
for the low-pass design

2 /1 in the pass-band

j
W(e ) = 1 in the stop-band

0 in the transition band.
This states that the optimization will control the ratio of the pass-band to stop-band ripple,
and that the transition will not contribute to the error criterion.
Let be a compact subset of the frequency band from 0 to representing the pass- and
stop-bands. The goal is to nd the set of lter parameters {ak }, k = 0, . . . , M/2 + 1 that
minimize the maximum value of the error E( ej ) over the pass- and stop-bands.

M/2

min max E( e ) j
j j
= min max W ( e ) Hd ( e ) ak cos(k)

over ak over over ak over
k=0
where is the disjoint set of frequency bands that make up the pass- and stop-bands of
the lter.
The solution is found by an iterative optimization routine. We do not attempt to cover the
details of the algorithm here, and merely note:
The method is based on reformulating the problem as one in polynomial approximation,
using Chebyshev polynomials, where
cos(k) = Tk (cos())
where Tk (x) is a polynomial of degree k, (see the Class Handout on Chebyshev lter
design). Consequently

M/2

M/2
j
H( e ) = ak cos(k) = ak (cos())k
k=0 k=0
177
The algorithm uses Chebyshevs alternation theorem to recognize the optimal solution.
In general terms the theorem is stated:
Dene the error E(x) as above, namely

E( ej ) = W ( ej ) Hd ( ej ) H( ej )
and the maximum error as

E( ej ) = argmaxx E( ej )
A necessary and sucient condition that H( ej ) is the unique Lth-order

polynomial minimizing E( ej ) is that E( ej ) exhibit at least L + 2
extremal frequencies, or alternations, that is there must exist at least L+2
values of , k , k = [0, 1, . . . , L + 1], such that 0 < 1 < . . . < L+1 ,
and such that

E( ej k ) = E( ej k+1 ) = E( ej ) .
Note that the alternation theorem is simply a way of recognizing the optimal equi
ripple solution. For example, the following gure is from a Parks-McClellan low-pass
lter with length M + 1 = 17.
| H ( e j w ) |
tr a n s itio n
1 + d 1
1 -d 1
p a s s -b a n d s to p -b a n d
a lte r n a tio n fr e q u e n c ie s
d 2
0
w
w c w s p
From above, H( ej ) is written as a polynomial of degree M/2,

M/2
j
H( e ) = ak (cos())k
k=0
so that L = M/2 and the pass- and stop-bands must exhibit at least M/2 + 2 = 10
points of alternation. These 10 points are shown in the gure.
The Parks-McClellan algoritm uses the Remez exchange optimization method.
178
See Proakis and Manolakis Sec. 10.2.4 or Openheim, Schafer and Buck Sec. 7.4 for details.
MATLAB Parks-McClellan Function: The Parks-McClellan algorithm is implemented

in the MATLAB function
b = firpm(M,F,A,W)
where b is the array of lter coecients, M is the lter order (M+1 is the length of the lter),
F is a vector of band edge frequencies in ascending order, A is a set of lter gains at the band
edges, and W is an optional set of relative weights to be applied to each of the bands.
For example, consider a band-pass lter with two pass-bands as shown below:
H d (e jw
)
p a s s
1
p a s s
0 .7
s to p s to p s to p
w
0 w 1 w 2 w 3 w 4 w 5 w 6 w 7 w 8 p
tr a n s itio n tr a n s itio n tr a n s itio n tr a n s itio n
There are ve distinct bands in this lter, separated by four transition regions. The lter
would require the following specications:
F = [0 1 2 3 4 5 6 7 8 1]
A = [0 0 1 1 0 0 0.7 0.7 0 0]
W = [10 1 10 1 10]
where the errors in the stop-bands have been weighted 10 times more heavily than in the
pass-bands. See the MATLAB help/documentation for more details.
Example 1
Design a length 33 Parks-McClellan band-pass lter with the following band
specications:
H d (e jw
)
p a s s
1 0
s to p s to p
w
0 0 .2 p 0 .4 p 0 .7 p 0 .8 5 p p
tr a n s itio n tr a n s itio n
179
Weight the stop-band ripple ten times more heavily than the pass-band.
Solution:
h=firpm(32,[0 0.2 0.4 0.7 0.85 1],[0 0 10 10 0 0],[10 1 10])

freqz(h,1)
5 0
M a g n itu d e ( d B )
-5 0
0
-1 0 0
0 .2 0 .4 0 .6 0 .8 1
N o r m a liz e d F r e q u e n c y (x p r a d /s a m p le )
1 0 0 0
5 0 0
P h a s e (d e g re e s )
-5 0 0
-1 0 0 0
-1 5 0 0
0 0 .2 0 .4 0 .6 0 .8 1
N o r m a liz e d F r e q u e n c y (x p r a d /s a m p le )
1710
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 181
Reading:
Proakis and Manolakis: 7.3.1, 7.3.2, 10.3
Oppenheim, Schafer, and Buck: 8.7.3, 7.1
1 FFT Convolution for FIR Filters

The response of an FIR lter with impulse response {hk } to an input {fk } is given by the
linear convolution

yn = fk hnk .
k=
The length of the convolution of two nite sequences of lengths P and Q is N = P + Q 1.

The following gure shows a sequence {fn } of length P = 6, and a sequence {hn } of length
Q = 4 reversed and shifted so as to compute the extremes of the convolved sequence y0 and
y8 .
f k
P = 6
5
k
0
n = 0 h n -k
Q = 4
n -k
-3 0
n = 8
h n -k
Q = 4
n -k
0 8
The convolution property of the DFT suggests that the FFT might be used to convolve two
equal length sequences
yn = IDFT {DFT {fn } .DFT {hn }} .
1 c D.Rowell 2008
copyright
181
However, DFT convolution is a circular convolution, involving periodic extensions of the two
sequences. The following gure shows the circular convolution of length 6, on two sequences
{fn } of length P = 6 and {hn } of length Q = 4. The periodic extensions cause overlap in
the rst Q 1 samples, generating wrap-around errors in the DFT convolution.
f k
5
k
0
P = 6
n = 0
h n -k
n -k
-3 0
Q = 4 o v e r la p o f Q - 1 s a m p le s
DFT convolution of two sequences of length P and Q (P Q) in DFTs of length P

1. Produces an output sequence of length P , whereas linear convolution produces
an output sequence of length P + Q 1.
2. Introduces wrap-around error in the rst Q1 samples of the output sequence.

The solution is to zero-pad both input sequences to a length N P +Q1
and then to use DFT convolution with the length N sequences.
For example, if {fn } is of length P = 237, and {hn } is of length Q = 125, for error-free
convolution we must perform the DFTs in length N 237 + 125 1 = 461. If the available
FFT routine is radix-2, we should choose N=512.
The use of the FFT for Filtering Long Data Sequences: The DFT convolution
method provides an attractive alternative to direct convolution when the length of the data
record is very large. The general method is to break the data into manageable sections, then
use the FFT to to perform the convolution and then recombine the output sections. Care
must be taken, however, to avoid wrap-around errors. There are two basic methods used for
convolving long data records. Let the impulse response {hn } have length Q.
Overlap-Save Method: (Also known as the overlap-discard, or select-savings method.)

In this method the data is divided into blocks of length P samples, but with successive
blocks overlapping by Q 1. The DFT convolution is done on each block with length
P , and wrap-around errors are allowed to contaminate the rst Q 1 samples of the
output. These initial samples are then discarded, and only the error-free P (Q 1)
samples are saved in the output record.
182
With the overlap of the data blocks, in the mth block the samples are
fm (n) = f (n + m(P (Q 1))), n = 0, . . . , P 1,
and after DFT convolution in length P , giving ymP (n), the output is taken as

ymP (n + (Q 1)), n = 0, . . . , P (Q 1)
ym (n) =
0, otherwise.
and the output is formed by concatenating all such records:

y(n) = ym (n m(P Q + 1)).
m=0
h (n )
n
Q
o v e r la p p in g in p u t s e c tio n s
f(n )
P
P
P
n
y (n ) d is c a r d o u tp u t s a m p le s in th is r e g io n
0 P
n
P
Q -1
y (n )
1 P d is c a r d o u tp u t s a m p le s in th is r e g io n
n
P
Q -1
y (n ) d is c a r d o u tp u t s a m p le s in th is r e g io n
2 P
n
P
Q -1
Overlap-Add Method: In this method the data is divided into blocks of length P , but
the DFT convolution is done in zero-padded blocks of length N = P + Q 1 so that
183
wrap-around errors do not occur. In this case the output is identical to the linear
convolution of the two blocks, with an initial rise of length Q 1 samples, and a
trailing section also of length Q 1 samples. It is easy to show that if the trailing
section of the mth output block is overlapped with the initial section of the (m + 1)th
block, the samples add together to generate the correct output values.
h (n )
n
Q
f(n )
P P P
n
y 1 (n )
a d d o u tp u t s a m p le s in th is r e g io n
n
P + Q - 1
y 2 (n )
a d d o u tp u t s a m p le s in th is r e g io n
n
P + Q - 1
y 3 (n )
n
P + Q - 1
MATLABs fftfilt() function performs DFT convolution using the overlap-add method.
2 The Design of IIR Filters

An IIR lter is characterized by a recursive dierence equation

N
M
yn = ak ynk + bk fnk
k=1 k=0
and a rational transfer function of the form

b0 z0 + b1 z 1 + . . . + bM z M
H(z) =
z0 + a1 z 1 + . . . + aN z N
IIR lters have the advantage that they can give a better cut-o characteristic than a FIR
lter of the same order, but have the disadvantage that the phase response cannot be well
controlled.
184
The most common design procedure for digital IIR lters is to design a continuous lter in
the s-plane, and then to transform that lter to the z-plane. Because the mapping between
the continuous and discrete domains cannot be done exactly, the various design methods are
at best approximations.
2.1 Design By Approximation of Derivatives:

Perhaps the simplest method for low-order systems is to use backward-dierence approxi
mation to continuous domain derivatives.
Example 1
Suppose we wish to make a discrete-time lter based on a prototype rst-order
high-pass lter
s
Hp (s) = .
s+a
The dierential equation describing this lter is
dy df
+ ay =
dt dt
The backward-dierence approximation to a derivative based on samples taken
at intervals T apart is
dx xn xn1

dt T
and substitution into the dierential equation gives
yn yn1 fn fn1
+ ayn =
T T
or
1 1
yn = yn1 + (fn fn1 )
1 + aT 1 + aT
1 z 1 z1
H(z) = 1
=
(1 + aT )1 + z (1 + aT )z + 1
This example indicates that the method uses the transformation

1 z 1
s
T
in Hp (s). For higher order terms
n
n 1 z 1
s .
T
185
Example 2
Convert the continuous low-pass Butterworth lter with c = 1 rad/s to a digital
lter with a sampling time T = 0.5 s. The transfer function is
1
Hp (s) = .
s2 + 2s + 1
The discrete-time transfer function is
1
H(z) = 1 2
1z
1z1
T
+ 2 T
+1
2
T
=
(1 + 2T + T ) (2 + 2T )z 1 + z 2
2
and with T = 0.5 s,

0.25
H(z) =
1.9571 2.7071z 1 + z 2
The frequency response of this lter is plotted in Example 4.
In general, the backward-dierence does not lead to satisfactory digital lters that mimic
the prototype lter characteristics. (See Proakis and Manolakis, Sec. 10.3.1).
2.2 Design by Impulse-Invariance:

In the impulse-invariant design method the impulse response {hn } of the digital lter is
taken to be proportional to the samples of the impulse response hp (t) of the continuous lter
Hp (s) with a sampling interval of T seconds. The most common form is
hn = T hp (nT ).
h (t)
d (t)
h (t)
d (t) P ro to ty p e
H (s ) t
t
s a m p le r h (n T )
186
Then

H(z) = T Z {hp (nT )} = T ZT L1 {Hp (s)}
since hp (t) = L1 {Hp (s)}, and where ZT {} indicates the z-transform of a continuous func
tion with sampling interval T .
Example 3
Find the impulse-invariant IIR lter from the prototype continuous lter
a
Hp (s) = .
s+a
Solution: Using Laplace transform tables

1 a
hp (t) = L = a eat .
s+a
and from z-transform tables
a
ZT a eat = .
1 eaT z 1
The IIR lter is
aT
H(z) = T ZT a eat =
1 eaT z 1
and the dierence equation is
yn = eaT yn1 + aT fn
For the digital lter

j T
H( e ) = H(z)|z= ej T = hk ej kT
k=0
and the DTFT of the samples of the continuous prototypes impulse response is

j kT 1 2k
DTFT {hp (nT )} = hp (kT ) e = Hp j .
k=0
T k=
Then if hn = T hp (nT ),

j T 2k
H( e )= Hp j .
k=
T
The discrete-time frequency response is therefore a superposition of shifted replicas of the

frequency response of the prototype. As a result, aliasing will be present in H( ej ) if the
prototypes frequency response |Hp (j )| = 0 for || /T .
187
H ( e jw )
o v e r la p p in g r e p lic a s
a lia s in g d is to r tio n
in th e p a s s - b a n d .
w
-2 p -p 0 p 2 p
For this reason the impulse-invariance method is not suitable for the design of high-pass or
band-stop lters, which by denition require a prototype Hp (s) with a non-zero frequency
response at = /T.
Example 4
Design an impulse-invariant lter based on the second-order low-pass Butter-
worth prototype used in Example 2, with T = 0.5 s.
1
Hp (s) =
s2 + 2s + 1
Solution: From z-transform tables

1 eaT sin(T )z
ZT L =
(s + a)2 + 2 z 2 + 2z eaT cos(T )z + e2aT
and Hp (s) may be written in this form
1
Hp (s) =
(s + 1/ 2)2 + (1/ 2)2

so that a = 1/ 2, and = 1/ 2. Substituting these values,
0.1719z 1
H(z) = T ZT L1 {Hp (s)} =
1 1.3175z 1 + 0.4935z 2
The frequency response of the impulse-invariant, and backward-dierence (from
Example 2) lters are compared with the prototype below:
188
1
x - P r o to ty p e B u tte r w o r th F ilte r
0 .9
0 .8
F r e q u e n c y r e s p o n s e m a g n itu d e
0 .7
0 .6
Im p u ls e - in v a r ia n t ( T = 0 .5 )
0 .5 B a c k w a r d d iffe r e n c e ( T = 0 .5 )
0 .4
0 .3
0 .2
0 .1
0
0 1 2 3 4 5 6
F re q u e n c y (ra d /s )
The MATLAB function

[bz, az] = impinvar(bs, as, Fs)
will compute the numerator az, and denominator bz coecients for an impulse-invariant
lter from the continuous prototype coecients bs and as, with a sampling frequency Fs.
The lter in Example 4 can be designed in a single line:
[bz, az] = impinvar(1, [1 sqrt(2) 1], 2).
189
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 191
Reading:
Proakis and Manolakis: Sec. 10.3.3
Oppenheim, Schafer, and Buck: Sec. 7.1
1 The Design of IIR Filters (continued)
1.1 Design by the Matched z-Transform (Root Matching)

Given a prototype continuous lter Hp (s),
M
k=1 (s zk )
Hp (s) = K N
k=1 (s pk )
with zeros zk , poles pk , and gain K, the matched z-transform method approximates the ideal
mapping
Hp (s) H(z)|z= esT
by mapping the poles and zeros
M
k=1 (z ezk T )
H(z) = K N
k=1 (z epk T )
where K must be determined from some empirical response comparison between the pro
totype and digital lters. Note that an implicit assumption is that all s-plane poles and
zeros must lie in the primary strip in the s-plane (that is |(s)| < /T ). Poles/zeros on the
s-plane imaginary axis will map to the unit circle, and left-half s-plane poles and zeros will
map to the interior of the unit circle, preserving stability.
p r im a r y
jW {z }
s - p la n e z - p la n e
s tr ip
jp /T x
o
x o
x o s x o {z }
x o
o
x
- jp /T
1 c D.Rowell 2008
copyright
191
The steps in the design procedure are:
1. Determine the poles and zeros of the prototype lter Hp (s).
2. Map the poles and zeros to the z-plane using z = esT .
3. Form the z-plane transfer function with the transformed poles/zeros.
4. Determine the gain constant K by matching gains at some frequency (for a low-pass
lter this is normally the low frequency response).
5. Add poles or zeros at z = 0 to adjust the delay of the lter (while maintaining causal
ity).
Example 1
Use the matched z-transform method to design a lter based on the prototype
rst-order low-pass lter
a
Hp (s) = .
s+a
Solution: The prototype has a single pole at s = a, and therefore the digital
lter will have a pole at z = eaT . The transfer function is
1
H(z) = K .
z eaT
To nd K , compare the low frequency gains of the two lters:
lim Hp (j ) = 1
0
K
lim H( ej ) = ,
0 1 eaT
therefore choose K = 1 eaT . Then
1 eaT (1 eaT )z 1
H(z) = =
z eaT 1 eaT z 1
yn = eaT yn1 + (1 eaT )fn1 .
Note that this is not a minimum delay lter, because it does not use fn . Therefore
we can optionally add a zero at the origin, and take
(1 eaT )z (1 eaT )
H(z) = =
z eaT 1 eaT z 1
as the nal lter design.
192
Example 2
Use the matched z-transform method to design a second-order band-pass lter
based on the prototype lter
s
Hp (s) =
s2 + 0.2s + 1
with a sampling interval T = 0.5 sec. Make frequency response plots to compare
the prototype and digital lters.
Solution: The prototype lter as a zero at s = 0, and a complex conjugate
pole pair at s = 0.1 j 0.995, so that
z1
H(z) = K
(z e(0.1+j 0.995)T )(z e(0.1j 0.995)T )
z1
= K 2
z 1.6718z + 0.9048
To nd K , compare the gains at = 1 rad/s (the peak response of Hp (j )),
|H (j )| = 5
p j T =1
H ( e ) = 10.54K .
=1
and to match the gains K = 5/10.54 = 0.4612, and
0.4612(z 1)
H(z) =
z2 1.6718z + 0.9048
4
F r e q u e n c y R e s p o n s e M a g n itu d e
1 p ro to ty p e
m a tc h e d z -tra n s fro m
0 0 1 2 3
F re q u e n c y (ra d /s )
4 5 6
193
To create a minimum delay lter, make the order of the numerator and denomi
nator equal by adding a zero at the origin,
0.4612z(z 1) 0.4612(1 z 1 )
H(z) = =
z 2 1.6718z + 0.9048 1 1.6718z 1 + 0.9048z 2
and implement the lter as
yn = 1.6718yn1 0.9048yn2 + 0.4612(fn fn1 ).
1.2 Design by the Bilinear Transform

As noted above, the ideal mapping of a prototype lter to the z-plane is
Hp (s) H(z)|z= esT
or
1
s ln(z)
T
so that
H(z) = Hp (s)|s= 1 ln(z) .
T
The Laurent series expansion for ln(z) is

3 5
z1 1 z1 1 z1
ln(z) = 2 + + + for {z} 0, z = 0.
z+1 3 z+1 5 z+1
The bilinear transform method uses the truncated series approximation

1 2 z1
s ln(z)
T T z+1
In a more general sense, any transformation of the form

z1 s+A
s=A which implies z =
z+1 sA
is a bilinear transform. In particular, when A = 2/T the method is known as

Tustins method.
With this transformation the digital lter is designed from the prototype using
H(z) = Hp (s)|s= 2 ( z1 )
T z+1
194
Example 3
Find the bilinear transform equivalent of an integrator
1
Hp (s) = .
s
Solution:

1 T 1 + z 1
H(z) = =
s s= 2 ( z1 ) 2 1 z 1
T z+1

T
yn = yn1 + (fn + fn1 )
2
which is the classical trapezoidal (or mid-point) rule for numerical integration.
The bilinear transform maps the left half s-plane to the interior of the unit circle, and thus
preserves stability. In addition, we will see below that it maps the entire imaginary axis of
the s-plane to the unit circle, and thus avoids aliasing in the frequency response.
jW {z } z - p la n e
s - p la n e
jW p
W T
s {z }
Thus every point on the frequency response of the continuous-time prototype lter, is mapped
to a corresponding point in the frequency response of the discrete-time lter, although with a
dierent frequency. This means that every feature in the frequency response of the prototype
lter is preserved, with identical gain and phase shift, at some frequency the digital lter.
Example 4
Find the bilinear transform equivalent of a rst-order low-pass lter
a
Hp (s) = .
s+a
195
Solution:

a
H(z) =
s + a s= 2 ( z1 )
T z+1
(aT /2)(z + 1)
=
(z 1) + (aT /2)(z + 1)
(aT /2)(1 + z 1 )
=
(1 + aT /2) (1 aT /2)z 1
1 aT /2 aT /2
yn = yn1 + fn .
1 + aT /2 1 + aT /2
Comparing the frequency responses of the two lters,

H( ej T )=0 = 1 0 = Hp (j 0)

lim H( ej T ) = 0 = lim Hp (j ),
/T 2
demonstrating the assertion above that the entire frequency response of the pro
totype lter has been transformed to the unit circle.
1.2.1 Frequency Warping in the Bilinear Transform

The mapping
2 z1
s
T z + 1
implies that when z = ej T ,

2 ej T
1 2 T
s= = j tan
T ej T + 1 T 2
so that
j T 2 T
H( e ) = Hp j tan
T 2
which gives a nonlinear warping of the frequency scales in the frequency response of the two
lters.
196
fr e q u e n c y in p r o to ty p e filte r
W p
2 ta n W T
T 2
-p /T
W
W p /T d
fr e q u e n c y in d ig ita l filte r
In particular
H( ej 0 ) = Hp (j 0) , and H( ej ) = Hp (j )
and there is no aliasing in the frequency response.
1.2.2 Pre-warping of Critical Frequencies in Bilinear Transform Filter Design

The specications for a digital lter must be done in the digital domain, that is the critical
band-edge frequencies must relate to the performance of the nal design - not the continuous
prototype.
Therefore, in designing the continuous prototype we need to choose band-edge frequencies
that will warp to the correct values after the bilinear transform. This procedure is known as
pre-warping. For example, if we are given a specication for a digital low-pass lter such as
| H ( jW ) |2
1
1
2
1 + e
1
1 + l 2
0
0 W c W r W (ra d /s e c )
197
we would pre-warp the frequencies c and r to

2 c T 2 r T
tan c =
, and r = tan
T 2 T 2
and design the prototype to meet the specications with c and c as the band edges.

Design Procedure: For any class of lter (band-pass, band-stop) the procedure is:
(1) Dene all band-edge critical frequencies for the digital lter.
(2) Pre-warp all critical frequencies using = (T /2) tan(T /2).
(3) Design the continuous prototype using the pre-warped frequencies.
(4) Use the bilinear transform to transform Hp (s) to H(z).
(5) Realize the digital lter as a dierence equation.
Example 5
Use the bilinear transform method to design a low-pass lter, with T = .01 sec.,
based on a prototype Butterworth lter to meet the following specications.
|H ( j2 p F ) | 2
1
1
2 = 0 .9
1 + e
1
2
= 0 .0 5
1 + l 0
0 1 0 2 0 F (H z )
Solution: Pre-warp the band-edges:

2 c T
c = tan = 64.9839 rad/s
T 2

2 r T
r = tan = 145.3085 rad/s.
T 2
From the specications = 0.3333 and = 4.358, and the required order for the
prototype Butterworth lter is
log(/)
N = 3.1946
log(r /c )
198
so take N = 4. The four poles (p1 , . . . , p4 ) lie on a circle of radius c 1/N =
82.526,
|pn | = 82.526,
pn = (2n + 3)/8
for n = 1 . . . 4. The prototype transfer function is

p1 p2 p3 p4
Hp (s) =
(s p1 )(s p2 )(s p3 )(s p4 )
5.3504 107
= 4 .
s + 223.4897s3 + 24974s2 + 1.6348 106 s + 5.3504 107
Applying the bilinear transform
H(z) = Hp (s)|s= 2 ( z1 )
T z+1
gives
0.0112(1 + z 1 )4
H(z) =
1.0000 1.9105z 1 + 1.6620z 2 0.6847z 3 + 0.1128z 4
and the frequency response of the digital lter (as a power gain) is shown below:
1
P o w e r R e s p o n s e |H ( j2 p F ) |2
0 .8
0 .6
0 .4
0 .2
0 F
0 1 0 2 0 3 0 4 0 5 0
F re q u e n c y (H z )
199
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 201
Reading:
Class Handout: Direct-Form Digital Filter Structures
Proakis and Manolakis: Sec. 9.1 9.3
Oppenheim, Schafer, and Buck: 6.0 6.5
1 Direct-Form Filter Structures

Linear shift-invariant digital lters can be represented in block diagram form in terms of the
three primitive elements
c o e ffic ie n t ( g a in ) u n it d e la y s u m m e r
a
fn y = a fn fn z -1 y = fn f y = fn - g
-
n n -1 n n n
g n
2 Transversal FIR Structure

Let the FIR structure to be implemented be

N
H(z) = bn z k
k=0
so that the dierence equation is

N
yn = bk fnk .
k=0
The following block diagram is the transversal form of this system:
f n -1 f n -2
f n -3 f
-1 -1 n -N
f n z z -1
z -1
z
b b b 2
b 3
b N
0 1
y n
1 c D.Rowell 2008
copyright
201
The following MATLAB code implements this structure in a point-by-point ltering function:
% ------------------------------------------------------------------------
% 2.161 Classroom Example - firdf - Demonstration FIR Direct Form
% implementation.
% Usage : 1) Initialization:
% b = [1 2 3 4 5 4 3 2 1];
% y = iirdf1(initial, b);
% where b are the numerator polynomial coefficients. Example:
% y = iirdf1(initial,[1 2 5 2 1]);
% Note: firdf returns y = 0 for initialization
% 2) Filtering:
% y_out = firdf(f);
% where f is a single input value, and
% y_out is the computed output value.
% Example: To compute the step response:
% for j=1:100
% y(j) = firdf(1);
% end
% ------------------------------------------------------------------------
%
function y_n = firdf(f_n,B)
persistent f_register Bx N
%
% The following is initialization, and is executed once
%
if (ischar(f_n) && strcmp(f_n,initial))
N = length(B);
Bx = B;
f_register = zeros(1,N);
y_n = 0;
else
% Filtering:
y_n = 0;
for J = N:-1:2
f_register(J) = f_register(J-1);
y_n = y_n + Bx(J)*f_register(J);
end
y_n = y_n + Bx(1)*f_n;
f_register(1) = f_n;
end
202
3 IIR Direct Form Structures

Let the IIR structure to be implemented be
N
k=0 bn z k
H(z) = N
1+ k=1 ak z k
where it is assumed that the orders of the numerator and denominator of H(z) are equal.
The dierence equation is

N
N
yn = ak ynk + bk fnk .
k=1 k=0
Write H(z) as a pair of cascaded sub-systems,

H(z) = H1 (z)H2 (z)
where

N
1
H1 (z) = bn z k , and H2 (z) = N .
k=0
1+ k=1 ak z k
3.1 Direct Form I

Dene an intermediate variable xn , and implement as X(z) = H1 (z)F (z) and Y (z) =
H2 (z)X(z), or in dierence equation form as

N
xn = bk fnk
k=0
N
yn = ak ynk + xn
k=1
as shown below:
b 0 x n
f n y n
-
z -1
z -1
b 1 a 1
f n -1 y n -1
-1
z z -1
b 2 a 2
f n -2 y n -2
-1
z z -1
b 3 a 3
f n -3 y n -3
-1
z z -1
b N a
N
f n -N y n -N
203
The following MATLAB code implements the Direct Form I structure in a point-by-point
ltering function.
% ------------------------------------------------------------------------
% 2.161 Classroom Example - iirdf1 - Demonstration IIR Direct Form I
% implementation.
% y = iirdf1(initial, b, a)
% where b, a are the numerator and denominator polynomial
% coefficients. Example:
% [b,a] = butter(7,0.4);
% y = iirdf1(initial,b,a);
% Note: iirdf1 returns y = 0 for initialization
% 2) Filtering:
% y_out = iirdf1(f_{in};
% where f_in is a single input value, and
% for j=1:100
% y(j) = iirdf1(1);
% end
% ------------------------------------------------------------------------
function y_n = iirdf1(f_n,B,A)
persistent f_register y_register Bx Ax N
%
%
N = length(A);
Ax = A;
Bx = B;
f_register = zeros(1,N);
y_register = zeros(1,N);
y_n = 0;
else
% Filtering: (Note that a Direct Form I filter needs two shift registers.)
x = 0; y = 0;
for J = N:-1:2
y_register(J) = y_register(J-1); % Move along the shift register
f_register(J) = f_register(J-1);
y = y - Ax(J)*y_register(J);
x = x + Bx(J)*f_register(J);
end
x = x + Bx(1)*f_n;
y_n = y + x;
f_register(1) = f_n;
y_register(1) = y_n;
end
204
3.2 Direct Form II
The Direct Form II structure results from reversing the order of H1 (z) and H2 (z) so that
X(z) = H2 (z)F (z) and Y (z) = H1 (z)X(z), or in dierence equation form as

N
xn = ak fnk
k=1

N
yn = bk xnk .
k=0
as shown below:
x b
n 0
x b 0
f n y n f n
n
y
- -
n
-1
z -1
z -1
z
a 1 b a
x n -1
1 1 x n -1 b 1
-1
z -1
z z -1
a 2 b 2 a b
2 x 2
x n -2
n -2
z -1
z -1
z -1
a 3 b 3 a b
3 3
x n -3
x n -3
-1
z z -1
z -1
a b N a b N
N
x n -N
N
x n -N
From the left hand gure it can be seen that the values xnk , k = 0, . . . N , in the two shift
registers is equal, and that they can be combined to create the Direct Form II structure, as
is shown on the right.
The following MATLAB code implements the Direct Form II structure in a point-by-point
ltering function:
% ------------------------------------------------------------------------
% 2.161 Classroom Example - iirdf2 - Demonstration IIR Direct Form II
% implementation.
% y = iirdf2(initial, b, a)
% [b,a] = butter(7,0.4);
% y = iirdf2(initial,b,a);
% 2) Filtering:
% y_out = iirdf2(f_{in};
205
% for j=1:100
% y(j) = iirdf2(1);
% end
% ------------------------------------------------------------------------
%
function y_n = iirdf2(f_n,B,A)
persistent register Bx Ax N
%
%
N = length(A);
Ax = A;
Bx = B;
register = zeros(1,N);
y_n = 0;
else
% Filtering: (Note that a Direct Form II filter needs only a single

% shift register.)
x = 0; y = 0;
for J = N:-1:2
register(J) = register(J-1); % Move along the shift register
x = x - Ax(J)*register(J);
y = y + Bx(J)*register(J);
end
x = x + f_n;
y_n = y + Bx(1)*x;
register(1) = x;
end
4 Transposed Direct Forms

The transposed forms result from the transposition theorem from signal-ow graph theory,
which states that in a signal-ow graph if
The arrows on all graph branches are reversed.
Branch points become summers, and summers become branch points.
The input and output are swapped,
then the input/output relationships remain unchanged. The same applies to block diagrams.
4.1 Transposed Transversal FIR Filter

The transposed FIR structure is shown below:
206
x b 0 b 0
f n
n
y f n y n
n
-1
z z -1
x n -1 b 1
b 1
-1
z
x b 2 -1
n -2 z x
b 2
z -1
b 3
x n -3 z -1
b 3
-1
z
b N
x n -N
-1
z
b N
T ra n s v e rs a l F IR T ra n s p o s e d T ra n s v e rs a l F IR
% ------------------------------------------------------------------------
% 2.161 Classroom Example - firtdf - Demonstration Transposed FIR Direct
% Form implementation.
% y = firtdf(initial, b)
% b = [1 2 3 4 5 4 3 2 1];
% y = firtdf(initial,b);
%
% Note: firtdf returns y = 0 for initialization
% 2) Filtering:
% y_out = firtdf(f_{in});
% for j=1:100
% y(j) = firtdf(1);
% end
% ------------------------------------------------------------------------
%
function y_n = firtdf(f_n,B)
persistent register Bx N
%
%
N = length(B);
Bx = B;
register = zeros(1,N-1);
207
y_n = 0;
else
% Filtering:
y_n = register(1) + Bx(1)*f_n;
% Update for the next iteration
for J = 1:N-2
register(J) = register(J+1) + Bx(J+1)*f_n;
end
register(N-1) = Bx(N)*f_n;
end
4.2 Transposed Direct Form II

The following diagram shows the result when the transposition theorem is applied to a Direct
Form II structure.
x b 0 b 0
n f
f n y n
n y n
-
-1
z z -1
a 1 x b 1
n -1
b 1 - a 1
-1
z
a x b 2 -1
2 n -2 z x
b 2 a 2
-
z -1
a 3 b 3
x n -3 z -1
b 3 a 3
-
-1
z
a b N
N
x n -N
-1
z
b N a N
-
D ir e c t F o r m II T r a n s p o s e d D ir e c t F o r m II
This block diagram simply reorganizes the dierence equation as

N
yn = b0 fn + (bk fnk ak ynk )
k=1
which is implemented in the MATLAB function iirtdf2() on the next page.
208
% ------------------------------------------------------------------------
% 2.161 Classroom Example - iirtdf2 - Demonstration Transposed IIR Direct
% Form II implementation.
% y = iirtdf2(initial, b, a)
% [b,a] = butter(7,0.4);
% y = iirtdf2(initial,b,a);
% 2) Filtering:
% y_out = iirtdf2(f_{in};
% for j=1:100
% y(j) = iirtdf2(1);
% end
% ------------------------------------------------------------------------
%
function y_n = iirtdf2(f_n,B,A)
persistent register Bx Ax N
%
%
N = length(A);
Ax = A;
Bx = B;
register = zeros(1,N-1);
y_n = 0;
else
% Filtering: (Note that a Transposed Direct Form II filter needs only a single
% register.) Also note that this is not strictly a shift register.
y_n = register(1) + Bx(1)*f_n;
% Update for the next iteration
for J = 1:N-2
register(J) = register(J+1) + Bx(J+1)*f_n - Ax(J+1)*y_n;
end
register(N-1) = Bx(N)*f_n - Ax(N)*y_n;
end
5 Coecient Sensitivity in Direct Form Filters

In practice high-order IIR Direct Form lters are rarely used because of the sensitivity of pole and
zero positions to small perturbations in the values of the coecients ak and bk in the dierence
209
equation. If the transfer function is

M 1
A(z) k=0 bk z
H(z) = = ,
B(z) 1+ Nk=1 ak z
1
and the poles are clustered near the unit circle, then small perturbations in any of the ak from
the desired value (perhaps because of nite precision limitations) may cause the lter to become
unstable.
To demonstrate this, consider a low-pass lter with
N
N

1

A(z) = 1 + ak z = 1 pk z 1
k=1 k=1
where the poles pk are within the unit circle, but close to z = 1, and write pk = 1 + k , where
|k | 1.
Now let a single (arbitrary) coecient ar be be perturbed by to
ar = ar +
so that the denominator polynomial becomes

N

A (z) = 1 + ak z 1 + z r .
k=1
As || increases, one or more of the poles may move outside the unit circle, leading to instability.
It is dicult to dene the general condition, but we can easily nd the condition that leads to a
pole migrating to z = 1, since then
A (1) = A(1) + = 0,
that is, there will be a pole at z = 1 if

= A(1),
or alternatively, if
N

= ((k).
k=1
Example 1
Consider a low-pass lter
1 1
H(z) = =
(1 0.99z 1 ) 1 2.97z 1 + 2.9403z 2 0.970299z 3
with three poles at z = 0.99. Find the perturbation allowed in any coecient
that will create a marginally stable system with a pole at z = 1. Discuss some
methods of decreasing the sensitivity.
2010
Solution: For the third-order system A(1) = 106 , so any change of =
A(1) = 106 in any coecient will move one of the poles from z = 0.99 to
z = 1. Any perturbation larger than this will generate an unstable lter.
Now consider the eect of implementing this lter as a cascade connection of two
lters, a second-order lter H1 (z), and a rst-order lter H2 (z), that is
1 1
H(z) = H1 (z)H2 (z) = .
(1 0.99z ) 1 z 1
1 2
with a pair of dierence equations

xn = 1.98xn1 0.9801xn2 + fn
yn = 0.99yn1 + xn .
For H1 (z), A1 (1) = 104 , while for H2 (z) A2 (1) = 102 and the sensitivity is
signicantly reduced.
If the lter is implemented as a cascade connection of three rst-order lters,
1 1 1
H(z) = H1 (z)H2 (z)H(3 z)(z) = . .
1 z 1 1 z 1 1 z 1
with a set of dierence equations
wn = 0.99wn1 + fn
vn = 0.99vn1 + wn
yn = 0.99yn1 + vn ,
for any of the rst-order sections Hk (z), A1 (1) = 102 , and the coecient
sensitivity is signicantly reduced even further.
This example demonstrates that the sensitivity to coecient precision can be often drastically
reduced by implementing a lter with low order sections.
5.1 Cascade Structures

If the transfer function is written in terms of its poles and zeros
M1 1
M2 1 1
K=1 (1 ek z ) K=1 (1 gk z )(1 g k z )
H(z) = N N
1 1 1
K=1 (1 ck z ) K=1 (1 dk z )(1 dk z )
1 2
where the ck and ek are real poles and zeros, and dk , dk and gk , g k are complex conjugate pole and
zero pairs, it is common to realize the system as a cascade chain of rst- and second-order sections
(usually Direct Form II):
A rst-order Direct Form II section, implementing a real pole ck and zero ek
1 ek z 1
Hk (z) = ,
1 ck z 1
as
yn = ck yn1 + fn ek fn1
is shown below
2011
x n
f n y n
-1
z
c k x n -1
-e k
A second-order Direct Form 2 section, implementing a conjugate pole pair
dk , dk = r ej
has a denominator polynomial
(1 dk z 1 )(1 dk z 1 ) = 1 2r cos()z 1 + r2 z 2
and when paired with a pair of zeros (either real, or a complex conjugate pair) to give a transfer
function
1 b1 z 1 + b2 z 2
H(z) =
1 2r cos()z 1 + r2 z 2
and dierence equation
yn = 2r cos()yn1 + r2 yn2 + fn + b1 fn1 + b2 fn2
is shown below
x n
f n y n
-1
z
2 r c o s (q ) x b 1
n -1
-1
z
-r 2 b 2
x n -2
Example 2
Implement the system
0.04756z 3 + 0.14273z 2 + 0.14273z + 0.04756

H(z) =
z 3 1.3146z 2 + 1.17043z 0.47524
as a set of cascaded rst- and second-order systems.
Solution: Factor the transfer function and rewite as
0.04756(1 + z 1 )3
H(z) = .
(1 0.6711z 1 + 0.7386z 2 )(1 0.6435z 1 )
2012
Implement the lter as a cascaded pair
0.04756(1 + 2z 1 + z 2 )
H1 (z) =
1 0.6711z 1 + 0.7386z 2
1 + z 1
H2 (z) =
1 0.6435z 1
with a pair of dierence equations
xn = 0.6711xn1 0.7386xn2 + 0.04756(fn + 2fn2 + fn2 )

yn = 0.6435yn1 + xn + xn1 .
There is a lot of exibility in choosing which zeros to associate with the poles of each low order
section, and how to distribute the overall gain between the sections. A general (rule-of-thumb)
procedure is
(1) Select the poles closest to the unit circle.
(2) Find the closest zeros to those poles.
(3) Combine into a second-order section.
(4) Repeat until all zeros are accounted for.
5.2 Parallel Structures

A high order lter may also be realized as a set of parallel second- and rst-order sections using
partial fractions, and implemented as
H(z) = H1 (z) + H2 (z) + H3 (z) . . . + HN (z)
so that
Y (z) = (H1 (z) + H2 (z) + H3 (z) . . . + HN (z)) F (z)
f n H 1 (z ) y n
H 2 (z )
H 3 (z )
H N (z )
2013
Example 3
Implement the system of Example 2 as a parallel realization
0.04756z 3 + 0.14273z 2 + 0.14273z + 0.04756

H(z) =
z 3 1.3146z 2 + 1.17043z 0.47524
Solution: Using a partial fraction expansion
0.2929 .0877z 0.2271
H(z) = 0.0476 + 2 .
z 0.6435 z 0.6711z + 0.7386
Implement as three sections
H1 (z) = 0.0476
0.2929z 1
H2 (z) =
1 0.6435z 1
.0877z 1 0.2271z 2
H3 (z) =
1 0.6711z 1 + 0.7386z 2
with dierence equations
un = 0.0476fn
vn = 0.6435vn1 + 0.2929fn1
wn = 0.6711wn1 0.7386wn2 + 0.0877fn1 0.2271fn2
yn = un + vn wn
2014
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 211
Reading:
Class Handout: Interpolation (Up-sampling) and Decimation (Down-sampling)
Proakis and Manolakis: Secs. 11.1 11.5, 12.1
Oppenheim, Schafer, and Buck: Sec. 4.6, Appendix A
Stearns and Hush, Ch. 9
1 Interpolation and Decimation
1.1 Up-Sampling (Interpolation) by an Integer Factor

Consider a data set {fn } of length N , where fn = f (nT ), n = 0 . . . N 1 and T is the
sampling interval. The task is to resample the data at a higher rate so as to create a new
data set {fn }, of length KN , representing samples of the same continuous waveform f (t),
sampled at intervals T /K. The following gure shows a cosinusoidal data set with N = 8
samples, and resampled with the same data set interpolated by a factor K = 4.
1 1
(a) original data set (b) interpolated data set
(8 samples) (32 samples)
0.5 0.5
0 0
0.5 0.5
1 1
2 4 6 8 10 20 30
1 c D.Rowell 2008
copyright
211
1.1.1 A Frequency Domain Method

This method is useful for a nite-sized data record. Consider the DFTs {Fm } and {Fm }
of a pair of sample sets {fn } and {fn }, both recorded from f (t) from 0 t < T , but with
sampling intervals T and T /K respectively. Let N and KN be the corresponding sample
sizes. It is assumed that T has been chosen to satisfy the Nyquist criterion:
Let F (j) = F {f (t)} be the Fourier transform of f (t), and let f (t) be sampled at
intervals T to produce f (t). Then

1

2n
F (j) = F j (1)
T n=0 T
is periodic with period 2/T , and consists of scaled and shifted replicas of F (j).
Let the total sampling interval be T to produce N = T /T samples.
If the same waveform f (t) is sampled at intervals T /K to produce f (t) the period
of its Fourier transform F (j) is 2K/T and

K

2Kn
F (j) = F j (2)
T n=0 T
which diers only by a scale factor, and an increase in the period. Let the total
sampling period be T as above, to generate KN samples.
We consider the DFTs to be sampled representations of a single period of F (j) and

F (j). The equivalent line spacing in the DFT depends only on the total duration
of the sample set T , and is = 2/T in each case:

2m
Fm = F j , m = 0, 1, . . . N 1
T

2m
Fm = F j , m = 0, 1, . . . KN 1.
T

From Eqs. (1) and (2) the two DFTs {Fm } and Fm are related:

KFm m = 0, 1, . . . , N/2 1

Fm = 0 m = N/2, . . . , N K N/2 1

KFm(K1)N m = N K N/2, . . . , KN 1
The eect of increasing N (or decreasing T ) in the sample set, while maintaining
T = N T constant, is to increase the length of the DFT by raising the eective
Nyquist frequency N .
N
N = =
T T
212
F
m
N u m b e r o f s a m p le s : N
S a m p lin g in te r v a l: D T
-N /2 0 N /2 N 3 N /2 2 N m
(a )
F
N u m b e r o f s a m p le s : 2 N
m
S a m p lin g in te r v a l: D T /2
0 N 2 N m
(b )
The above gure demonstrates these eects by schematically, by comparing the DFT of (a)
a data set of length N derived by sampling at intervals T , and (b) a data set of length
2N resulting from sampling at intervals T /2. The low frequency region of both spectra
are similar, except for a scale factor, and the primary dierence lies in the high frequency
region, centered around the Nyquist frequency, in which all data points are zero.
The above leads to an algorithm for the interpolation of additional points into a data
set, by a constant factor K:
1. Take the DFT of the original data set to create {Fm } of length N .
2. Insert (K 1)N zeros into the center of the DFT to create a length KN array.
3. Take the IDFT of the expanded array, and scale the sequence by a factor K.
1.1.2 A Time-Domain Method

We now examine an interpolation scheme that may implemented on a sample-by-sample
basis in real-time using time domain processing alone. As before, assume that the process
f (t) is sampled at intervals T , generating a sequence {fn } = {f (nT )}. Now assume that
K 1 zeros are inserted between the samples to form a sequence {fk } at intervals T /K.
f 3
n
0 n
0 1 2 3 4 5 6 7
(a) Initial data set, N=8
f 3
n
2
0 n
0 5 10 15 20
(b) Data set with two samples interpolated between samples, N=24
213
This is illustrated above, where a data record with N = 8 samples has been expanded by
a factor K = 3 to form a new data record of length N = 24 formed by inserting two zero
samples between each of the original data points. We now examine the eect of inserting
K 1 samples with amplitude 0 after each sample. The DFT of the original data set is

N 1
2mn
Fm = fn ej N , m = 0...N 1
n=0
and for the extended data set {fn }, n = 0 . . . KN 1

KN 1
2mn
Fm = fn ej KN , m = 0 . . . KN 1
n=0
However, only the original N samples contribute to the sum, so that we can write

N 1
2mk
Fm = fKk ej N
k=0
= Fm , m = 0 . . . KN 1
since fKk = fk . We note that {Fm } is periodic with period N , and {Fm } is periodic with
period KN , so that {Fm } will contain K repetitions of {Fm }.
^
F m lo w - p a s s filte r
p a s s b a n d r e p lic a tio n s o f F
N y q u is t
fre q u e n c y
0 K N /2 K N
w = K p /D T w = 2 K p /D T
lo w - p a s s
d ig ita l filte r
^
F m
0 K N /2 K N
w = K p /D T w = 2 K p /D T
The magnitude of the DFTs of the two waveforms is shown above. The eect of inserting the
K 1 zeros between the original samples has been to generate a waveform with an equivalent
sampling interval of T /K s, and a Nyquist frequency of K/T rad/s. The line resolution
is unchanged, and the original DFT {Fm } is replicated K times within the frequency span
of 2K/T rad/s.
214
F 10
m
8
2
m
0
0 1 2 3 4 5 6 7
(a) DFT of Initial data set, N=8
^
F 10
m
8
2
m
0
0 5 10 15 20
(b) DFT of data set with two samples interpolated between samples, N=24
The fully interpolated waveform may be reconstructed by elimination of the replications

of the original spectral components. While this might be done in the frequency domain,
the most common method is to low-pass lter the padded data sequence to retain only the
base-band portion of the spectrum as shown below.
D T /K
D T
~ ^
f(t) A n ti- a lia s in g f(t) S a m p le r K {fn } L o w - p a s s d ig ita l
lo w - p a s s filte r filte r
In s e rt K -1 z e ro s
F ( jW ) H ( jW ) = 0 , |W |> p /D T F ( jW ) H ( jW ) D T
{ fn } in te r p o la te d
b e tw e e n s a m p le s H (z )
w a v e fo rm
c o n tin u o u s d o m a in d is c r e te d o m a in
D T /K
1.2 Down-Sampling (Decimation) by an Integer Factor

Decimation by an integer factor K is the reverse of interpolation, that is increasing the
sampling interval T by an an integer factor (or decreasing the sampling frequency). At
rst glance the process seems to be simple: simply retain every Kth sample from the data
sequence so that fn = fnK where {fn } is the down-sampled sequence, as shown in below.
D T L D T
t t
D o w n - s a m p le r
Caution must be taken however to prevent aliasing in the decimated sequence. It is not valid
to directly down-sample a sequence directly unless it is known a-priori that the spectrum of
the data set is identically zero at frequencies at and above the Nyquist frequency dened by
the lower sampling frequency.
215
In discussing sampling of continuous waveforms, we described the use of a pre-aliasing
lter to eliminate (or at least signicantly reduce) spectral components that would introduce
aliasing into the sampled data set. When down-sampling, the digital equivalent is required:
a digital low-pass lter is used to eliminate all spectral components that would cause aliasing
in the resampled data set. The complete down-sampling scheme is:
D T D T L D T
t t
D ig ita l lo w - p a s s D o w n - s a m p le r
a n ti- a lia s in g filte r L
1.3 Resampling with a non-integer factor
Assume that the goal is to re-sample a sequence by a non-integer factor p that can be
expressed as a rational fraction, that is
N
P =
M
where N and M are positive integers. This can be achieved by (1) interpolation by a factor
N , followed by (2) decimation by a factor M , as shown below.
{ fn } u p - s a m p le r in te r p o la tio n filte r a n ti- a lia s in g filte r d o w n - s a m p le r ^

{ fm }
N H u p (z ) H d n (z ) M
in te r p o la tio n d e c im a tio n
However, since the two low-pass lters are cascaded, they may be replaced with a single lter
with a cut-o frequency that is the lower of the two lters, as is shown below
u p - s a m p le r lo w - p a s s filte r d o w n - s a m p le r
{ fn } ^
N { fm }
H lp (z ) M
2 Introduction to Random Signals

In dealing with physical phenomena and systems we are frequently confronted with non
deterministic, (stochastic, or random) signals, where the temporal function can not be de
scribed explicitly, nor predicted. Some simple examples are wind loading on a structure,
additive noise in a communication system, and speech waveforms. In this brief examina
tion of random phenomena we concentrate on common statistical descriptors of stochastic
waveforms and input-output relationships of linear systems excited by random waveforms.
L in e a r s y s te m
n o n - d e r te r m in is tic in p u t n o n - d e r te r m in is tic o u tp u t
216
Since we cannot describe f (t), we must use statistical descriptors that capture the essence
of the waveform. There are two basic methods of doing this:
(a) Describe the waveform based on temporal measurements, for example dene the mean
of the waveform as

1 T /2
= lim f (t) dt
T T T /2
(b) Conjecture an ensemble of random processes fi (t), i = 1, . . . N , with identical statistics

and dene the descriptors my measurements made across the ensemble at a given time,
for example

(t) = lim fi (t).
N
N =1
A stationary process is dened as one whose ensemble statistics are indepen

dent of time.
An ergodic process is one in which the temporal statistics are identical to the
ensemble statistics.
Clearly, ergodicity impies stationarity.
tim e - b a s e d s ta tis tic s d e r iv e d fr o m a s in g le r a n d o m p ro c e s s

f1 (t)
f2 (t)
f3 (t)
e n s e m b le - b a s e d s ta tis tic s d e r iv e d fr o m a h y p o th e tic a l c o lle c tio n o f

r a n d o m p r o c e s s e s w ith id e n tic a l d e s c r ip to r s
In practice statistical descriptors are usually derived experimentally from measurements.

For example, the mean of of a waveform might be estimated from a set of 1000 samples of
a waveform and computed as
1
1000
1 =
fi .
1000 i=1
But is not the mean, it is simply an estimator of the true mean. If we repeated the
experiment, we would come up with a dierent value 2 . In statistical descriptions we use
the terms expected value, or expectation, designated E {x}, and say
i } =
E {
to indicate that our experimental estimates
i will be clustered around the true mean .
217
2.1 Ensemble Based Statistics

2.1.1 The Probability Density Function (pdf )
The pdf is strictly an ensemble statistic that describes the distribution of samples x across
the amplitude scale. Its denition is
1
p(xa ) = lim Prob {xa x xa + x}
x0 x
so that the probability that a single sample lies in the range a x b is

b
Prob {a x b} = p(x) dx,
a
and we note

Prob { x } = p(x) dx = 1.

For an ergodic process, the pdf can also be described from a single time series
x (t) x
a re a = P ro b {a x b }
b b
a a
t p (x )
and may be interpreted as the fraction of time that the waveform dwells in the range
b f (t) b.
Two common pdfs are
(a) The Uniform distribution a random sample taken from a uniformly distributed ran
dom process is equally likely to be found anywhere between a minimum and maximum
value.
1
axb
p(X) = ba
0 elsewhwere.
p (x )
1
b - a
x
a b
218
(b) The normal (or gaussian) distribution] The normal distribution denes the well known
bell-shaped-curve of elementary statistics
1 (x)2
p(x) = e 22
2
where is the mean of the distribution, and 2 is the variance.

p (x )
x
m - s m m + s
Note: The central limit theorem of statistics states that any random process that is
the sum of a large number of underlying independent random processes, regardless
of their distributions will be described by a gaussian distribution.
Many ensemble based statistical descriptors may be described in terms of the pdf, for example
The mean

E {x} = = xp(x)dx

The variance
2
2

E (x ) = = (x )2 p(x)dx

and by expanding the integral

2 2

2

= x p(x)dx 2 xp(x)dx + p(x)dx = E x2 2

Example 1
Find the mean and variance a random variable that is uniformly distributed
between x0 and x0 + .
The pdf is
1
x0 x x0 +
p(X) =
0 elsewhere.
219
The mean is

x0+
x
= xp(x)dx = dx = x0 +
x0 2
which is the mid-point of the range.
The variance is
1
2 = E x2 2 = 2
12
2.2 Time-based Statistics

Two stochastic waveforms may have identical pdfs (and hence equal means and variances),
but be very dierent qualitatively, for example
f1 (t)
f2 (t)
t
t o to + t
These two waveforms obviously dier in
spectral content, or
self-similarity between themselves at a time t0 and some time later.
A random waveform cannot be predicted exactly at any time, but clearly in the above gure
the upper waveform (with greater high frequency spectral content) has less self-similarity, or
correlation with itself, after a delay of .
The correlation functions are a measure of the degree to which the value of a function
depends upon its past. For innite duration waveforms the auto-correlation function f f ( )
is dened as
1 T /2
f f ( ) = lim f (t)f (t + ) dt
T T T /2
2110
and is a measure of the self-similarity of the function f (t) at time t and at a time later.
The cross-correlation function f g ( ) measures the similarity between two dierent functions
f (t) and g(t) at two times apart.
T /2
1
f g ( ) = lim f (t)g(t + ) dt
T T T /2
Note that these denitions must be modied for nite duration waveforms, and if f (t) exists
in the interval T1 t T2 , we dene
T2
f f ( ) = f (t)f (t + ) dt
T1
2111
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 221
Reading:
Proakis and Manolakis: Secs. 12,1 12.2
Oppenheim, Schafer, and Buck:
Stearns and Hush: Ch. 13
1 The Correlation Functions (continued)

In Lecture 21 we introduced the auto-correlation and cross-correlation functions as measures
of self- and cross-similarity as a function of delay . We continue the discussion here.
1.1 The Autocorrelation Function

There are three basic denitions
(a) For an innite duration waveform:

T /2
1
f f ( ) = lim f (t)f (t + ) dt
T T T /2
which may be considered as a power based denition.
(b) For an nite duration waveform: If the waveform exists only in the interval t1
t t2 t2
f f ( ) = f (t)f (t + ) dt
t1
which may be considered as a energy based denition.
(c) For a periodic waveform: If f (t) is periodic with period T

1 t0 +T
f f ( ) = f (t)f (t + ) dt
T t0
for an arbitrary t0 , which again may be considered as a power based denition.
1 c D.Rowell 2008
copyright
221
Example 1
Find the autocorrelation function of the square pulse of amplitude a and duration
T as shown below.
f(t)
t
0 T
The wave form has a nite duration, and the autocorrelation function is
T
f f ( ) = f (t)f (t + ) dt
0
The autocorrelation function is developed graphically below
f(t)
t
0 T
f(t+ t )
-t
t
0 T -t
r ff (t )
2
a
r ff (t ) = a 2
(T -t )
0
t
-T T
T
f f ( ) = a2 dt
0
= a2 (T | |) T T
= 0 otherwise.
222
Example 2
Find the autocorrelation function of the sinusoid f (t) = sin(t + ).
Since f (t) is periodic, the autocorrelation function is dened by the average over
one period
1 t0 +T
f f ( ) = f (t)f (t + ) dt.
T t0
and with t0 = 0

2/
f f ( ) = sin(t + ) sin((t + ) + ) dt
2 0
1
= cos(t)
2
and we see that f f ( ) is periodic with period 2/ and is independent of the
phase .
1.1.1 Properties of the Auto-correlation Function

(1) The autocorrelation functions f f ( ) and f f ( ) are even functions, that is
f f ( ) = f f ( ), and f f ( ) = f f ( ).
(2) A maximum value of f f ( ) (or f f ( ) occurs at delay = 0,

|f f ( )| f f (0), and |f f ( )| f f (0)
and we note that
f f (0) = f 2 (d) dt

is the energy of the waveform. Similarly
1
f f (0) = lim f 2 (t) dt
T T
is the mean power of f (t).
(3) f f ( ) contains no phase information, and is independent of the time origin.

(4) If f (t) is periodic with period T , f f ( ) is also periodic with period T .
(5) If (1) f (t) has zero mean ( = 0), and (2) f (t) is non-periodic,
lim f f ( ) = 0.

223
1.1.2 The Fourier Transform of the Auto-Correlation Function
Consider the transient case

Rf f (j ) = f f ( ) ej d

= f (t)f (t + ) dt ej d

j t
= f (t) e dt. f () ej d

= F (j )F (j )
= |F (j )|2
or
F
f f ( ) Rf f (j ) = |F (j )|2
where Rf f () is known as the energy density spectrum of the transient waveform f (t).
Similarly, the Fourier transform of the power-based autocorrelation function, f f ( )

f f (j ) = F {f f ( )} = f f ( ) ej d

T /2
1
= lim f (t)f (t + ) dt ej d
T T T /2
is known as the power density spectrum of an innite duration waveform.
From the properties of the Fourier transform, because the auto-correlation function
is a real, even function of , the energy/power density spectrum is a real, even
function of , and contains no phase information.
1.1.3 Parsevals Theorem

From the inverse Fourier transform

2 1
f f (0) = f (t) dt = Rf f (j ) d
2
or
1
2
f (t) dt = |F (j )|2 d,
2
which equates the total waveform energy in the time and frequency domains, and which is
known as Parsevals theorem. Similarly, for innite duration waveforms
T /2
2 1
lim f (t) dt = (j ) d
T T /2 2
equates the signal power in the two domains.
224
1.1.4 Note on the relative widths of the Autocorrelation and Power/Energy
Spectra
As in the case of Fourier analysis of waveforms, there is a general reciprocal relationship
between the width of a signals spectrum and the width of its autocorrelation function.
A narrow autocorrelation function generally implies a broad spectrum
f ff (t ) F ff ( jW )
b r o a d a u to c o r r e la tio n n a rro w s p e c tru m
t W
and a broad autocorrelation function generally implies a narrow-band waveform.

n a r r o w a u to c o r r e la tio n b ro a d s p e c tru m
t W
In the limit, if f f ( ) = ( ), then f f (j ) = 1, and the spectrum is dened to be white.

" w h ite " s p e c tr u m

im p u ls e a u to c o r r e la tio n 1
t W
1.2 The Cross-correlation Function

The cross-correlation function is a measure of self-similarity between two waveforms f (t)
and g(t). As in the case of the auto-correlation functions we need two denitions:

1 T /2
f g ( ) = lim f (t)g(t + ) d
T T T /2
in the case of innite duration waveforms, and

f g ( ) = f (t)g(t + ) d

for nite duration waveforms.
225
Example 3
Find the cross-correlation function between the following two functions
f(t) g (t)
T T
a a
t t
0 T 1 0 T 2
In this case g(t) is a delayed version of f (t). The cross-correlation is
r fg
(t )
2
a
0
t
T 2 - T 1
where the peak occurs at = T2 T1 (the delay between the two signals).
1.2.1 Properties of the Cross-Correlation Function

(1) f g ( ) = gf ( ), and the cross-correlation function is not necessarily an even function.
(2) If f g ( ) = 0 for all , then f (t) and g(t) are said to be uncorrelated.
(3) If g(t) = af (t T ), where a is a constant, that is g(t) is a scaled and delayed version of
f (t), then f f ( ) will have its maximum value at = T .
Cross-correlation is often used in optimal estimation of delay, such as in echolocation (radar,

sonar), and in GPS receivers.
226
Example 4
In an echolocation system, a transmitted waveform s(t) is reected o an object
at a distance R and is received a time T = 2R/c sec. later. The received signal
r(t) = s(tT )+n(t) is attenuated by a factor and is contaminated by additive
noise n(t).
R
tr a n s m itte d
s (t)
w a v e fo rm r e fle c tin g o b je c t
a s (t-T ) v e lo c ity o f p r o p a g a tio n : c

r e c e iv e d
r(t)
w a v e fo rm + d e la y T = 2cR
n (t)

sr ( ) = s(t)r(t + ) dt

= s(t)(n(t + ) + s(t T + )) dt

= sn ( ) + ss ( T )
and if the transmitted waveform s(t) and the noise n(t) are uncorrelated, that is
sn ( ) 0, then
sr ( ) = ss ( T )
that is, a scaled and shifted version of the auto-correlation function of the trans
mitted waveform which will have its peak value at = T , which may be used
to form an estimator of the range R.
1.2.2 The Cross-Power/Energy Spectrum

We dene the cross-power/energy density spectra as the Fourier transforms of the cross-
correlation functions:

Rf g (j ) = f g ( ) ej d

f g (j ) = f g ( ) ej d.

Then

Rf g (j ) = f g ( ) ej d

= f (t)g(t + ) ej dt d

j t
= f (t) e dt g() ej d

227
or
Rf g (j ) = F (j )G(j )
Note that although Rf f (j ) is real and even (because f f ( ) is real and even, this is
not the case with the cross-power/energy spectra, f g (j ) and Rf g (j ), and they are
in general complex.
2 Linear System Input/Output Relationships with Random In

puts:
Consider a linear system H(j ) with a random input f (t). The output will also be random
f(t) y (t)
H ( jW )
f ff ( t) f (t)
y y
Then
Y (j ) = F (j )H(j ),
Y (j )Y (j ) = F (j )H(j )F (j )H(j )
or
yy (j ) = f f (j ) |H(j )|2 .
Also
F (j )Y (j ) = F (j )F (j )H(j ),
or
f y (j ) = f f (j )H(j ).
Taking the inverse Fourier transforms

yy ( ) = f f ( ) F 1 |H(j )|2
f y ( ) = f f ( ) h( ).
3 Discrete-Time Correlation
Dene the correlation functions in terms of summations, for example for an innite length
sequence
f g (n) = E {fm gm+n }
N

1
= lim fm gm+n ,
N 2N + 1
m=N
and for a nite length sequence

N

f g (n) = fm gm+n .
m=N
228
The following properties are analogous to the properties of the continuous correlation func
tions:
(1) The auto-correlation functions ( f f (n) and f f (n) are real, even functions.
(2) The cross-correlation functions are not necessarily even functions, and
f g (n) = gf (n)
(2) f f (n) has its maximum value at n = 0,
|f f (n)| f f (0) for all n.
(3) If {fk } has no periodic component
lim f f (n) = 2f .
n
(4) f f (0) is the average power in an innite sequence, and f f (n) is the total energy in a
nite sequence.
The discrete power/energy spectra are dened through the z-transform

f f (z) = Z {f f (n)} = f f (n)z n
n=
and
f f (n) = Z 1 {f f (z)}

1
= f f (z)z n1 dz
2j
/T
T
= f f ( ej T ) ej nT d.
2 /T
Note on the MATLAB function xcorr(): In MATLAB the function call phi =
xcorr(f,g) computes the cross-correlation function, but reverses the denition of
the subscript order from that presented here, that is it computes
N N
1 1
f g (n) = fn+m gm = fn gnm
M N M N
where M is a normalization constant specied by an optional argument. Care must

therefore be taken in interpreting results computed through xcorr().
229
3.1 Summary of z-Domain Correlation Relationships

(The following table is based on Table 13.2 from Stearns and Hush)
Property Formula

Power spectrum of {fn } f f (z) = f f (n)z n
n=

Cross-power Spectrum f g (z) = f g (n)z n = gf (z 1 )
n=
1
Autocorrelation f f (n) = f f (z)z n1 dz
2j
1
Cross-correlation f g (n) = f g (z)z n1 dz
2 j
2 1
Waveform power E fn = f f (0) = f g (z)z 1 dz
2j
Linear system properties Y (z) = H(z)F (z)
yy (z) = H(z)H(z 1 )f f (z)
f y (z) = H(z)f f (z)
2210
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 231
Reading:
Proakis and Manolakis: Secs. 14.1 14.2
Oppenheim, Schafer, and Buck: 10.6 10.8
Stearns and Hush: 15.4, 15.6
1 Non-Parametric Power Spectral Density Estimation

In Lecture 22 we dened the power-density spectrum f f (j ) of an innite duration, real
function f (t) as the Fourier transform of its autocorrelation function f f ( )

1 T /2
f f (j ) = F lim f (t)f (t + ) dt .
T T T /2
with units of (physical-units)2 .s or (physical-units)2 /Hz, where physical-units are the units
of f (t), for example volts. The waveform power contained in the spectral region between
1 < || < 2 is
1 2
1
P = f f (j ) d + f f (j ) d
2 2 1

1 2
= f f (j ) d
1
since f f (j ) is a real, even function.

Similarly, we dened the energy-density spectrum Rf f (j ) of a real nite duration wave
form f (t) of duration T as the Fourier transform of its energy based autocorrelation function
f f ( )
T
Rf f (j ) = F f (t)f (t + ) dt .
0
with units of (physical-units) .s or (physical-units)2 .s/Hz, again where physical-units are

2 2
the units of f (t).

In this lecture we address the issue of estimating the PSD (power spectral density)
f f (j ) of an innite process using a nite length sample of the process. PSD analysis is an
important tool in engineering analysis. The practical problem is to form reliable estimates of
1 c D.Rowell 2008
copyright
231
the PSD from nite records of an innite random process. For example, the following gure
f f (j )
shows a stationary random process with three possible nite records. An estimator
of f f (j ) is to made from one of the nite-length records.
f(t)
R e c o rd 1 R e c o rd 2 R e c o rd 3
We ask ourselves about the statistics of estimators derived from the dierent records, in
particular,
(1) the bias in the estimator

B f f (j ) = f f (j ) E f f (j )
(2) the variance of the estimator

2

V f f (j ) = E
f f (j ) f f (j )
1.1 The Periodogram

1.1.1 The Continuous Periodogram
If f (t) is a stationary, real, random process, its autocorrelation function is dened by the
ensemble statistics
f f ( ) = E {f (t)f (t + )} .
For an ergodic process, if we have a single record of duration T we can compute an estimator,
f f ( ) based on the time average:
T /2
1
f f ( ) = f (t)f (t + ) dt,
T T /2
and
f f ( ) = lim f f ( ).
T
232
Furthermore, the Fourier transform of f f ( ) provides an estimator
f f (j ) of the PSD
T /2
f f (j ) =
f f ( ) ej d
T /2
T /2
T /2
1
= f (t)f (t + ) dt ej d
T T /2 T /2
1
= F (j )F (j )
T
1
= |F (j )|2 .
T
where F (j ) is the Fourier transform of the nite data record. The periodogram estimator
IT (j ) is then dened as
f f (j ) = 1 |F (j )|2 .
IT (j ) =
T
1.1.2 The Discrete-Time Periodogram

For an innite sampled data record {fn }, the autocorrelation function is dened by the
expectation
f f (m) = E {fn fn+m } ,
and, invoking ergodicity, a time-average denition is
1
N 1
f f (m) = lim fn fn+m .
N N
n=0
As in the continuous case, we can use a single nite length record, of length N , to form
an estimator of f f (j ). The equivalent periodogram denition (through the DFT) is
1 1
f f (k) =
IN (k) = Fk Fk = |Fk |2 .
N N
where {Fk } is the DFT of {fn }.
1.1.3 The Relationship between IT (j ) and IN (k)

We frequently want to used discrete-time analysis to estimate the PSD of a continuous
waveform. Consider a nite sample set {fn }, of length N , derived from sampling a continuous
waveform f (t) with sampling interval , so that
fn = f (n), n = 0, . . . , N 1.
From sampling theory F (j ), the Fourier transform of the sampled waveform f (t) is
F (j ) = F (j ),
233
and through the DTFT

N 1
F (j ) = fn ej n .
n=0
Then the continuous periodogram, with record length T = N , evaluated at = 2k/N is

2
j 2k 1 j 2k
IT = F
T T N
2
1 j 2k
= F
N N
and since N
j 2k 1

F = fn ej 2kn/N = Fk
N n=0

j 2k
IT = |Fk |2 = IN (k)
T N
The discrete-time periodogram is therefore a scaled version of the continuous periodogram.
1.1.4 Properties of the Periodogram:

(1) The periodogram is a real, even function. Because it is dened by the Fourier
transform of the autocorrelation function (which is a real, even function), the peri
odogram is also a real, even function.
(2) The periodogram is a biased estimator. Consider the denition of the estimate
of the autocorrelation implicitly used above
1
N 1

f f (m) = fn fn+m .
N n=0
With a nite length record the overlapping region of the two records {fn } and {fn+m }
in the summation only includes N |m| terms:
fn
n
0 N -1
N -|m | te rm s
fn + m
N -m -1
n
-m N -1
234
The estimated autocorrelation function used to compute the priodogram is therefore

biased. For the mth lag, the unbiased time-average estimator of the autocorrelation
function should therefore be
1
N 1
f f (m) = fn fn+m , m = (N 1), . . . , N 1
N |m| n=0
which is sometimes known as the mean-lagged-product. Then

N 1
N |m|
E {IN (k)} = f f (m) ej 2km/N
N
m=(N 1)
The periodogram is therefore a biased estimator of f f (j ). We note, however, that

lim IT (j ) = f f (j )
T
and that IT (j ) is therefore asymptotically unbiased.

(3) Variance of the periodogram. A somewhat surprising result is that the variance of
the of IN (k) (and IT (j )) does not decrease signicantly as N increases. For large
values of the lag m, particularly when m N , because there are few values in the
2k

sum the variance of the estimate f f (m) becomes large, and although f f j N is
asymptotically unbiased, the variance does not decay to zero as N . In fact
lim var[ f f (j )] 2 (j )
ff
N
that is the variance is approximately equal to the square of the true value (see OS&B
10.6, P&M 14.1.2), and is not reduced by taking a longer data record.
As more points are taken into the computation of the periodogram, the apparent
spectral resolution is increased, but the reliability of the extra points is marred by
the residual variance. The following gure shows periodograms of a 200 Hz sinusoid
in noise, sampled with = 0.001 s, and computed with N = 128, 512, and 2048
samples. (Only one side of the periodogram is shown). Notice that the variance does
not decrease.
0
N = 1 2 8
P S D (d B )
-5 0
-1 0 0
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0 4 0 0 4 5 0 5 0 0
0
N = 5 1 2
P S D (d B )
-5 0
-1 0 0
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0 4 0 0 4 5 0 5 0 0
0
N = 2 0 4 8
P S D (d B )
-5 0
-1 0 0
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0 4 0 0 4 5 0 5 0 0
235
(4) Implicit windowing in the periodogram. We noted above that
N
1
N |m|
E {IN (j )} = f f (m) emjT
N
m=(N 1)
The periodogram is therefore the Fourier transform of the true autocorrelation function
multiplied by the Bartlett window function
N |m |
wN (m) = , m = 0, . . . , N 1.
N
The periodogram is therefore the convolution of the true spectral density and the
Fourier transform of the Bartlett window, resulting in a smoothing operation, and the
introduction of Gibbs eect ripple adjacent to transitions in the spectrum.
Aside: It is interesting to note that for the Barlett window, F {wN (m)} 0 and as
a result the convolution maintains the requirement that IN (j ) 0.
1.2 Variance Reduction in Periodogram PSD Estimation

There are two common methods of reducing the variance in the periodogram method of PSD
estimation (1) the averaging of periodograms, and (2) the smoothing of a single periodogram
by windowing the autocorrelation function.
1.2.1 The Bartlett Method: The Averaging of Periodograms

Bartlett proposed the ensemble averaging of basic periodograms as a means of reducing the
variance in the estimate - at the expense of spectral resolution, and increased bias. Suppose
we have a record {fn } of length N , and we divide the total record into Q smaller blocks, each
(q)
of length K. Then if we compute the periodogram of each block and designate it IK (k),
q = 1, . . . , Q, the averaged periodogram may be dened as
Q
1 (q)
I K (k) = I (k)
Q q=1 K
Because each of the smaller blocks contains fewer samples, the spectrum computed through
the DFT will have decreased spectral resolution. However, provided each of the Q peri
odograms are statistically independent, the variance will be reduced by a factor of Q
1 (q) 1
V I K (j ) = V IK (j ) 2f f (k).
Q Q
1.2.2 The Welch Method: The Averaging of Modied Periodograms

Welch proposed two enhancements to Bartletts method:
(1) Welch showed that instead of dividing the data sequence into contiguous smaller blocks,
it is possible to overlap adjacent blocks by as much as 50% and still maintain statistical
independence. The eect is to increase Q, the number of data blocks in the ensemble
average and thus eect a further reduction in the variance of I K (k).
236
(2) Welchs method also applies a window function w(n) to each of the data records before
the DFT is computed. A modied periodogram based on the windowed record is then
computed
K1
(q) 1
IK (k) = fn w(n) ej 2nk/Q q = 1, . . . , Q
KU n=0
where
K1
1 2
U= w (n)
K n=0
is a factor to compensate for the fact that the windowing operation has reduced the
power of the waveform, and allows the estimator to be asymptotically unbiased.
As before, the spectral estimator is taken as the ensemble average of the windowed and
overlapped blocks
Q
1 (q)
I K (k) = I (k)
Q q=1 K
1.2.3 The Blackman-Tukey Method: Smoothing the Periodogram

Blackman and Tukey proposed that an eective method of variance reduction would be to
smooth the periodogram estimate with a low-pass lter, and that the smoothing operation
could be implemented by applying a suitable windowing function in the delay domain of the
autocorrelation function, to achieve the desired frequency domain convolution.
An alternative rationale is based on the reliability (variance) of the samples in the au
tocorrelation function for large lags. As was demonstrated at the start of this lecture, for
a xed length data record, the overlap in the product for computing f f (m) is N |m|,
and as |m| N the variance becomes large. The windowing operation serves to reduce the
contribution of these unreliable estimates in the computation of the PSD.
The Blackman-Tukey estimator is
M
1
IN (k) = f f (m)w(m) ej 2km/(2M 1)
m=(M 1)
where the window function is real and symmetric about its mid-point (to ensure that IN (k)
is real). The window length parameter M may be shorter than the data record length N .
The Blackman-Tukey estimate is therefore
IN (k) = IN (k) W (k)
where {W (k)} = DFT {w(n)}.

The choice of window function should be made to ensure that IN (j ) > 0. Many com
monly used windows, such as the Hamming and Hann windows, do not have this property,
and may result in negative values for the spectral estimates. The triangular Bartlett window
does maintain the sign of the estimates.
237
1.2.4 MATLAB Examples
MATLAB has built-in functions for spectral estimation, in particular the function spectrum()
is a powerful general function for non-parametric estimation, and the function pwelch() can
be used for Welchs method.
Notes:
(1) The MATLAB functions can represent a continuous periodogram IT (j ) by specifying

a sampling rate.
(2) The MATLAB default convention is that if the sample set is real the PSD is computed
as a one-sided spectrum, that is it is assumed the the power is contained in positive
frequencies only. Because of the real, even nature of the periodogram, the one-sided
spectrum has values twice those of the two-sided spectra. For complex data sets the
convention is to compute the two-sided spectra. The defaults can be overuled by
optional arguments in the function calls.
The following script was used to display a periodogram and a Welch estimate of a 200
Hz sinusoid in noise. A 1.024 sec. data record, with a sampling rate of 1000 samples/sec. is
simulated.
% Create the data record.
Fs = 1000;
t = 0:1/Fs:1.024;
f = cos(2*pi*t*200) + randn(size(t)); % A cosine of 200Hz plus noise
% Periodogram
figure(1);
h = spectrum.periodogram;
psd(h,f,Fs,Fs);
% Welchs method.
% Use default of 8 sections, 50% overlap, Hamming window
figure(2);
pwelch(f,128,64,128,Fs);
The following two plots were generated.
238
P e r io d o g r a m P o w e r S p e c tr a l D e n s ity E s tim a te
0
-5
P o w e r/fre q u e n c y (d B /H z ) -1 0
-1 5
-2 0
-2 5
-3 0
-3 5
-4 0
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0 4 0 0 4 5 0 5 0 0
W e lc h P o w e r S p e c tr a l D e n s ity E s tim a te
0
-5
P o w e r/fre q u e n c y (d B /H z )
-1 0
-1 5
-2 0
-2 5
-3 0
-3 5
-4 0
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0 4 0 0 4 5 0 5 0 0
239
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 241
Reading:
Class Handout: MATLAB Examples of Least-Squares FIR Filter Design
1 Least-Squares Filter Design

We now look at a FIR lter design technique that is based on experimental data
" d e s ir e d " o u tp u t
{d n }
in p u t +
F IR filte r {g } e rro r
{fn }
n
{e }
H (z ) - n
o u tp u t
{g n }
b filte r
c o e ffic ie n ts
k
{fn } {e }
le a s t- s q u a r e s n
d e s ig n a lg o r ith m
Given an input sequence {fn }, and a desired ltered output sequence {dn } , the task is to
design a FIR lter
M
1
H(z) = bk z k
k=0
that will minimize the error {en } = {dn }{gn } in some sense, where {gn } is the lter output.
In particular, we will look at a lter design method that minimizes the mean-squared-error
(MSE), where

MSE = E e2n

= E (dn gn )2

= E d2n + E gn2 2E {dn gn }
1 c D.Rowell 2008
copyright
241

Fall Term 2008
Lecture 241
Reading:
Class Handout: MATLAB Examples of Least-Squares FIR Filter Design
1 Least-Squares Filter Design

We now look at a FIR lter design technique that is based on experimental data
" d e s i r e d " o u t p u t
{d n }
in p u t +
F IR filte r {g } e rro r
{fn }
n
{e }
H (z ) - n
o u tp u t
{g n }
b filte r
c o e ffic ie n ts
k
{fn } {e }
d e s i g n a l g o r i t h m
Given an input sequence {fn }, and a desired ltered output sequence {dn } , the task is to
design a FIR lter

M 1
H(z) = bk z k
k=0
that will minimize the error {en } = {dn }{gn } in some sense, where {gn } is the lter output.
In particular, we will look at a lter design method that minimizes the mean-squared-error
(MSE), where

MSE = E e2n

= E (dn gn )2

= E d2n + E gn2 2E {dn gn }
1 c D.Rowell 2008
copyright
241
and in terms of correlation functions
MSE = dd (0) + gg (0) 2dg (0) .
For the FIR lter with coecients bk , the output is
{gn } = {fn } {bn }
and from the input/output properties of linear systems (Lec. 22)

gg (n) = Z 1 H(z)H(z 1 ) f f (n)

M
1 M 1
= bm bn f f (m n).
m=0 n=0
Similarly,
dg (n) = E {dm gn+m }

M 1

= E dm bk fn+mk
k=0

M 1
= bk E {dm fn+mk }
k=0

M 1
= bk f d (k n)
k=0
and

M 1
dg (0) = bk f d (k)
k=0
The MSE is therefore

M
1 M 1
M 1
MSE = dd (0) + bm bn f f (m n) 2 bk f d (k).
m=0 n=0 k=0
We now seek the optimum coecients.
MSE
M 1
=2 bm f f (m n) 2f d (n)
bn m=0
and setting MSE/bn = 0 for n = 0, 1, . . . , M 1, we have

M 1
bm f f (m n) = f d (n)
m=0
242
which is a set of linear equations in the coecients bk :
b0 f f (0) + b1 f f (1) + . . . + bM 1 f f (M 1) = f d (0)

b0 f f (1) + b1 f f (0) + . . . + bM 1 f f (M 2) = f d (1)
b0 f f (2) + b1 f f (1) + . . . + bM 1 f f (M 3) = f d (2) .
.
. ..
. .
b0 f f ((M 1)) + b1 f f ((M 2)) + . . . + bM 1 f f (0) = f d (M 1)
Recognizing that the auto-correlation function is an even function (f f (n) = f f (n)), we

can write the equations in matrix form

f f (0) f f (1) f f (2) f f (M 1) b0 f d (0)
f f (M 2)
f f (1) f f (0) f f (1) b0 f d (1)
. .. .. = ..
.
. . . .
f f (M 1) f f (M 2) f f (M 3) f f (0) b0 f d (0)
or
Rb = P
where
f f (0) f f (1) f f (M 1)
f f (2)
f f (1) f f (0) f f (M 2)
f f (1)

R= .. ..
. .
f f (M 1) f f (M 2) f f (M 3) f f (0)
is the correlation matrix,
T
b= b0 b1 b2 bM 1
are the lter coecients, and
T
P= f d (0) f d (1) f d (2) f d (M 1)
is the cross-correlation matrix.

The MSE FIR lter coecients are
b = R1 P.
Stearns and Hush show that with these coecients
(MSE)min = dd (0) PT b = dd (0) PT R1 P.
R is a Toeplitz matrix, and ecient algorithms (Levinson-Durbin see Proakis and

Manolakis Sec. 12.4.1) exist for its inversion (O(n2 ).
The development above requires that the processes {fn and dn are stationary.
243
1.1 A Simple MATLAB Tutorial Least-Squares Filter Design Func

tion
%-----------------------------------------------------------------------
function [B,MSE] = LSQFilt(f,d,M)
%LSQFilt - Demonstration routine for Least-Squares FIR filter design
%[B,MSE] = LSQFilt(f,d,M)
% f - rowvector of data samples - length N
% d - row vector of desired values - length N
% M - filter order
% Returns:
% B - vector of optimal filter coefficients
% MSE - minimized value of the mean-square-error
%
% Note: This routine is for tutorial purposes only. The Levinson method for
% toeplitz matrix inversion would be used in practical methods.
%
% Author: D. Rowell
% Revised: 10/29/07
%-----------------------------------------------------------------------
N = length(f);
% Compute the correlation coefficients.
% Note that matlab defines the cross-correlaton backwards!! and
% we need to reverse the order of the subscripts.
%
phiff=xcorr(f);
phifd=xcorr(d,f);
%
% Extract out the central regions (low-lag values) and form
% the autocorrelation matrix.
%
rff=phiff(N:N+M-1);
R = toeplitz(rff);
P=phifd(N:N+M-1);
%
% Compute the optimal filter coefficients
%
B=inv(R)*P;
%
% and the residual mean-square-error
%
phidd=xcorr(d);
MSE=phidd(N) - P*B;
%
%-----------------------------------------------------------------------
244
1.2 Application Examples of Least-Squares Filters
1.2.1 The Linear Predictor
Suppose that we want a lter that will predict the value of a process {fn } steps in the
future,
lin e a r p r e d ic to r
fn g = f n + D
H (z )
n
The structure of least-squares lter design is

a d v a n c e
z
D
+ d n
lin e a r p r e d ic to r g n
fn
H (z ) - e n
where dn = fn+ . This system is not realizable however (because the advance block in the
forward path is itself a predictor), and a practical design structure uses a delay of steps
in the forward path so that the lter design algorithm uses the input history to predict the
current value.
+ d n
- D
f n - D lin e a r p r e d ic to r g n -
fn z e
H (z )
n
d e la y
Once the lter is designed it may be used without the delay to predict the future. The most
common form is the one-step ( = 1 predictor.
Example 1
Stearns and Hush (p. 346) solve the problem of a one-step linear predictor for
an sinusoidal input function

2n
sn = sin , n = 0, 1, 2, . . .
12
T
and show that b = (3) 1 .
The following MATLAB code de4signs the lter using the function LSQFilt()
described above:
245
% One-step linear predictor for a sinusoidal input.
% Define a time vector, and the input vector:
t = 0:199;
s = sin(2*pi*t/12);
% In this case the desired output is the input
d = s;
% To make it causal we must delay the input to the filter
f = zeros(1,200);
f(2:200) = s(1:199);
% Compute the filter coefficients for a first-order filter
[B1,MSE] = LSQFilt(f,d,2)
% Repeat with a second-order model
[B2,MSE] = LSQFilt(f,d,3)
and produces the results

1.73205080756888e + 000
B1 =
1.00000000000000e + 000

1.73205080756897e + 000
B2 = 1.00000000000016e + 000
112.354570092066e 015
which agree with the closed-form solutions in Stearns and Hush.
1.2.2 System Identication

Suppose we have an unknown FIR system and the task is to determine its impulse response.
We can construct an experiment, using white noise to excite the system and record the input
and output series. The least-squares lter design method is then used to determine the
coecients of a FIR are used as estimates of the plant impulse response.
in p u t
re s p o n s e
{fn } U n k n o w n s y s te m
{d }
+ n
F IR filte r {g n
}
H (z ) -
b filte r
c o e ffic ie n ts
k
{fn } {e }
d e s i g n a l g o r i t h m
246
Example 2
Use a least-squares lter to estimate the impulse response of an unknown FIR
system with impulse response
Unknown Impulse Response

6
0
0 2 4 6 8 10 12
% System ID Using LSQFilt
% Create a FIR filter as the "unknown" plant
h = [0 1 2 3 4 5 4 3 2 1 0 0 0];
figure(1); stem(0:length(h)-1,h);
title(Unknown Impulse Response);
%
f = randn(1,1000);
% create output data representing the exerimental
% measurements
y = filter(h,1,f);
%
% Estimate the impulse response from the data
[h_opt,MSE] = LSQFilt(f,y,15);
figure(2); stem(0:length(h)-1,h_opt(1:length(h)));
title(Estimated Impulse Response);
giving the following result:
247
Estimated Impulse Response

6
0
0 2 4 6 8 10 12
1.2.3 Channel Compensation

Suppose a waveform has been corrupted by passing through an unknown LTI lter H(z),
and it is desired to recover the original waveform by using a cascade compensating inverse
lter Hc (z) = 1/H(z) so that Hc (zH (z) = 1. The structure is shown below, and includes an
empirically chosen delay element z to ensure causality of the compensating lter.
"u n k n o w n " c h a n n e l
in p u t
{fn } c h a n n e l re s p o n s e
-D H (z ) Y (z ) = H (z )F (z )
z
{d } = {f }
F IR c o m p e n s a tio n +
n n -D
{g }
f i l t e r
H c ( z ) -
c o m p e n s a te d o u tp u t
f i l t e r G (z ) = H c (z )H (z )F (z )
b
c o e ffic ie n ts
k
{ f n } {e }
d e s ig n a lg o r ith m
Example 3
Use LSQFilt(), with white noise as the input, to design a compensation lter
for an unknown recursive lter.
0.296
H(z) = .
1 1.721z 1 + 0.8187Z 2
248
Plot the resulting impulse response of the compensated channel.
Solution: The following MATLAB scipt uses a lter length M = 10, and a
delay factor = 1.
% Channel compensation using Least-Squares filter design.
% Input the numerator B(z), and denominator A(z) of the "unknown" plant
B_u = [0.2796 0];
A_u = [1 -1.7211 0.8187];
unknown = tf(B_u, A_u ,1);
h_unknown = impulse(unknown);
figure(1), stem(0:length(h_unknown)-1, h_unknown)
title(Unknown system impulse response);
% Use white noise as the input signal:
s = randn(1,200);
f = lsim(unknown,s);
% Create the desired output
$ (delay by one step so that resulting filter is causal)
d = zeros(1,200);
d(2:200) = s(1:199);
[B_noise,MSE] = LSQFilt(f,d,10);
figure(2), stem(B)
title(Filter coefficients found by using white noise as the input)
h_compensated = conv(h_unknown, B);
figure(3), stem(0:length(h_compensated)-1, h_compensated)
title(Compensated system impulse response)
The output is shown below.
Unknown system impulse response

0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.1
0.2
0.3
0 10 20 30 40 50 60
249
Filter coefficients found by using white noise as the input

4
8
1 2 3 4 5 6 7 8 9 10
Compensated system impulse response

1.2
0.8
0.6
0.4
0.2
0.2
0 10 20 30 40 50 60 70
Example 4
The following is another example of channel compensation, this time where the
channel is corrupted by an echo. For clarity we use a simple strong echo, so
that the waveform is yn = fn + 0.9fn3 . White noise is used as the excitation,
and a lter length M = 50, and a delay = 4 are chosen. Notice that complete
echo suppression is not possible with a nite length FIR lter.
% Model the environment as a nonrecursive filter:
2410
b = [1 0 0 0.9 0 0];
% Use white noise as the excitation
s = randn(1,200);
f = filter(b,1,s);
% The desired output is a delayed version of the input:
d = zeros(1,200); d(4:200) = s(1:197);
% Design the filter
[B,MSE] = LSQFilt(f,d,50);
figure(1), stem(0:length(B)-1, B)
title(Reverberation cancelling impulse response)
% Find the overall compensated system impulse response
h_comp = conv(B,b);
figure(2), stem(0:length(h_comp)-1, h_comp)
title(Compensated reverberation impulse response)
Reverberation cancelling impulse response

1
0.8
0.6
0.4
0.2
0.2
0.4
0.6
0.8
0 10 20 30 40 50
2411
Compensated reverberation impulse response

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.1
0 10 20 30 40 50 60
2412
MIT OpenCourseWare
http://ocw.mit.edu

Fall 2008

Fall Term 2008
Lecture 251
Reading:
Class Handout: Introduction to Least-Squares Adaptive Filters
Class Handout: Introduction to Recursive-Least-Squares (RLS) Adaptive Filters
Proakis and Manolakis: Secs. 13.1 13.3
1 Adaptive Filtering
In Lecture 24 we looked at the least-squares approach to FIR lter design. The lter coef
cients bm were generated from a one-time set of experimental data, and then used subse
quently, with the assumption of stationarity. In other words, the design and utilization of
the lter were decoupled.
We now extend the design method to adaptive FIR lters, where the coecients are
continually adjusted on a step-by-step basis during the ltering operation. Unlike the static
least-squares lters, which assume stationarity of the input, adaptive lters can track slowly
changing statistics in the input waveform.
The adaptive structure is shown in below. The adaptive lter is FIR of length M with
coecients bk , k = 0, 1, 2, . . . , M 1. The input stream {f (n)} is passed through the lter
to produce the sequence {y(n)}. At each time-step the lter coecients are updated using
an error e(n) = d(n) y(n) where d(n) is the desired response (usually based of {f (n)}).
d n
c a u s a l lin e a r F IR filte r y n -
+ e rro r
f n e n
H (z )
filte r c o e ffic ie n ts
A d a p tiv e L e a s t- S q u a r e s
A lg o r ith m
The lter is not designed to handle a particular input. Because it is adaptive, it can adjust
to a broadly dened task.
1 c D.Rowell 2008
copyright
251
1.1 The Adaptive LMS Filter Algorithm
1.1.1 Simplied Derivation
In the length M FIR adaptive lter the coecients bk (n), k = 1, 2, . . . , M 1, at time step n
are adjusted continuously to minimize a step-by-step squared-error performance index J(n):
2

M 1
J(n) = e2 (n) = (d(n) y(n))2 = d(n) b(k)f (n k)
k=0
J(n) is described by a quadratic surface in the bk (n), and therefore has a single minimum.
At each iteration we seek to reduce J(n) using the steepest descent optimization method,
that is we move each bk (n) an amount proportional to J(n)/b(k). In other words at step
n + 1 we modify the lter coecients from the previous step:
J(n)
bk (n + 1) = bk (n) (n) , k = 0, 1, 2, . . . M 1
bk (n)
where (n) is an empirically chosen parameter that denes the step size, and hence the rate
of convergence. (In many applications (n) = , a constant.) Then
J(n) e2 (n) e(n)

= = 2e(n) = 2e(n)f (n k)
bk bk bk
and the xed-gain FIR adaptive Least-Mean-Square (LMS) lter algorithm is
bk (n + 1) = bk (n) + e(n)f (n k), k = 0, 1, 2, . . . M 1
or in matrix form
b(n + 1) = b(n) + e(n)f (n),
where
b(n) = [b0 (n) b1 (n) b2 (n) bM 1 ]T
is a column vector of the lter coecients, and
f (n) = [f (n) f (n 1) f (n 2) f (n (M 1))]T
is a vector of the recent history of the input {f (n)}.

A Direct-Form implementation for a lter length M = 5 is
f(n ) -1 -1 -1 z -1
z z z
X X X X X d (n )
+ + + + + y (n )
+ + + +
b (n ) e (n )
b 0 (n ) b 1 (n ) b 2 (n ) b 3 (n ) 4
A d a p tiv e L M S A lg o r ith m
b k
(n + 1 ) = b k (n ) + L e (n )f(n -k )
252
1.1.2 Expanded Derivation

A more detailed derivation of the LMS algorithm (leading to the same result) is given in the
class handout Introduction to Least-Squares Adaptive Filters, together with a brief discussion
of the convergence properties.
1.1.3 A MATLAB Tutorial Adaptive Least-Squares Filter Function

% ------------------------------------------------------------------------
% 2.161 Classroom Example - LSadapt - Adaptive Lleast-squares FIR filter
% demonstration
% y = LSadapt(initial, Lambda, FIR_N)
% where Lambda is the convergence rate parameter.
% FIR_N is the filter length.
% Example:
% [y, e] = adaptfir(initial, .01, 51);
% Note: LSadapt returns y = 0 for initialization
% 2) Filtering:
% [y, b] = adaptfir(f, d};
% where f is a single input value,
% d is the desired input value, and
% y is the computed output value,
% b is the coefficient vector after updating.
%
% Version: 1.0
% Author: D. Rowell 12/9/07
% ------------------------------------------------------------------------
%
function [y, bout] = LSadapt(f, d ,FIR_M)
persistent f_history b lambda M
%
%
if (ischar(f) && strcmp(f,initial))
lambda = d;
M = FIR_M;
f_history = zeros(1,M);
b = zeros(1,M);
b(1) = 1;
y = 0;
else
% Update the input history vector:
for J=M:-1:2
f_history(J) = f_history(J-1);
253
end;
f_history(1) = f;
% Perform the convolution

y = 0;
for J = 1:M
y = y + b(J)*f_history(J);
end;
% Compute the error and update the filter coefficients for the next iteration
e = d - y;
for J = 1:M
b(J) = b(J) + lambda*e*f_history(J);
end;
bout=b;
end
1.1.4 Application Example - Suppression of Narrow-band Interference in a

Wide-band Signal
Consider an adaptive lter application of suppressing narrow band interference, or in terms
of correlation functions we assume that the desired signal has a narrow auto-correlation
function compared to the interfering signal.
Assume that the input {f (n)} consists of a wide-band signal {s(n)} that is contaminated
by a narrow-band interference signal {r(n)} so that
f (n) = s(n) + r(n).
The ltering task is to suppress r(n) without detailed knowledge of its structure. Consider
the lter shown below:
n a r r o w - b a n d
i n t e r f e r e n c e
r n
d n
fn fn -D c a u s a l lin e a r F IR filte r y n -
+ e rro r
- D
s n Z e n s n
w i d e - b a n d
H (z )
s ig n a l d e l a y
filte r c o e ffic ie n ts
A d a p tiv e L e a s t- S q u a r e s
A lg o r ith m
This is similar to the basic LMS structure, with the addition of a delay block of time steps
in front of the lter, and the denition that d(n) = f (n). The overall ltering operation is
a little unusual in that the error sequence {e(n)} is taken as the output. The FIR lter is
used to predict the narrow-band component so that y(n) r(n), which is then subtracted
from d(n) = f (n) to leave e(n) s(n).
The delay block is known as the decorrelation delay. Its purpose is to remove any
cross-correlation between {d(n)} and the wide-band component of the input to the lter
254
{s(n )}, so that it will not be predicted. In other words it assumes that
ss ( ) = 0, for | | > .
This least squares structure is similar to a -step linear predictor. It acts to predict the cur
rent narrow-band (broad auto-correlation) component from the past values, while rejecting
uncorrelated components in {d(n)} and {f (n )}.
If the LMS lter transfer function at time-step n is Hn (z), the overall suppression lter
is FIR with transfer function H(z):
E(z) F (z) z Hn (z)F (z)

H(z) = =
F (z) F (z)

= 1 z Hn (z)
= z 0 + 0z 1 + . . . + 0z (1) b0 (n)z b1 (n)z (+1) + . . .

. . . bM 1 (n)z (+M 1)
that is, a FIR lter of length M + with impulse response h (k) where

1 k=0

h (k) = 0 1k<

bk (n) k M +1
and with frequency response

M
+1
j
H(e ) = h (k)ejk .
k=0
The lter adaptation algorithm is the same as described above, with the addition of the
delay , that is
b(n + 1) = b(n) + e(n)f (n ))
or
bk (n + 1) = bk (n) + e(n)f ((n ) k), k = 0, 1, 2, . . . M 1.
Example 1
The frequency domain Characteristics of an LMS Suppression Filter:
This example demonstrates the lter characteristics of an adaptive LMS lter af
ter convergence. The interfering signal is comprised of 100 sinusoids with random
phase and random frequencies between 0.3 and 0.6. The signal is white noise.
The lter used has M = 31, = 1, and was adjusted to give a reasonable
convergence rate. The overall system H(z) = 1 z Hn (z) frequency response
magnitude is then computed and plotted, along with the z-plane pole-zero plot.
255
% The frequency domain filter characteristics of an interference

% suppression filter with finite bandwidth interference
%
% Create the interference as a closely packed sum of sinusoids
% between 0.3pi < Omega < 0.6pi with random frequency and phase
phase = 2*pi*rand(1,100);
freq = 0.3 + 0.3*rand(1,100);
f = zeros(1,100000);
for J=1:100000
f(J) = 0;
for k = 1:100
f(J) = f(J) + sin(freq(k)*J + phase(k));
end
end
% The "signal" is white noise

signal = randn(1,100000);
f = .005*f + 0.01*signal;
% Initialize the filter with M = 31 , Delta =1
% Choose filter gain parameter Lambda = 0.1
Delta = 1; Lambda = 0.5; M = 31;
x = LSadapt(initial,Lambda, M);
% Filter the data
f_delay = zeros(1,Delta+1);
y = zeros(1,length(f));
e = zeros(1,length(f));
for J = 1:length(f)
for K = Delta+1:-1:2
f_delay(K) = f_delay(K-1);
end
f_delay(1) = f(J);
[y(J),b] = LSadapt(f_delay(Delta+1),f(J));
e(J) = f(J) - y(J);
end;
% Compute the overall filter coefficients
% H(z) = 1 - z^{-Delta}H_{LMS}(z)
b_overall = [1 zeros(1,Delta-1) -b];
% Find the frequency response
[H,w] = freqz(b_overall,1);
zplane(b_overall,1)
The following plots show (i) the input input and output spectra, the lter fre
quency response magnitude, and (iii) the pole-zero plot of the lter. Note that
the zeros have been placed over the spectral region (0.3 < < 0.6) to create the
band-reject characteristic.
256
Spectrum of input signal f(n) Spectrum of output signal e(n)
8 8
7 7
6 6
5 5
Magnitude
Magnitude
4 4
3 3
2 2
1 1
0 0
0 1 2 3 0 1 2 3
Normalized angular frequency Normalized angular frequency
Adaptive Filter Frequency Response

5
5
Magnitude (dB)
10
15
20
25
30
0 0.5 1 1.5 2 2.5 3
Normalized frequency
Adaptive Filter zplane pole/zero plot
0.8
0.6
0.4
Imaginary Part
0.2
31
0
0.2
0.4
0.6
0.8
1
1 0.5 0 0.5 1
Real Part
257
Example 2
Suppression of a Sliding Sinusoid Superimposed on a Voice Signal:
In this example we demonstrate the suppression of a sinusoid with a linearly
increasing frequency superimposed on a voice signal. The ltering task is to task
is to suppress the sinusoid so as to enhance the intelligibility of the speech. The
male voice signal used in this example was sampled at Fs = 22.05 kHz for a
duration of approximately 8.5 sec. The interference was a sinusoid

Fs 2
r(t) = sin((t)) = sin 2 t + t
150
where Fs = 22.05 kHz is the sampling frequency. The instantaneous angular
frequency (t) = d(t)/dt is therefore
(t) = 2(50 + 294t) rad/s
which corresponds to a linear frequency sweep from 50 Hz to approx 2550 Hz

over the course of the 8.5 second message. In this case the suppression lter
must track the changing frequency of the sinusoid.
% Suppression of a frequeny modulated sinusoid superimposed on speech.

% Read the audio file and add the interfering sinusoid
[f,Fs,Nbits] = wavread(crash);
for J=1:length(f)
f(J) = f(J) + sin(2*pi*(50+J/150)*J/Fs);
end
wavplay(f,Fs);
% Initialize the filter

M = 55; Lambda = .01; Delay = 10;
x = LSadapt(initial, Lambda, M);
y = zeros(1,length(f));
e = zeros(1,length(f));
b = zeros(length(f),M);
f_delay = zeros(1,Delay+1);
% Filter the data
for J = 1:length(f)
for K = Delta+1:-1:2
f_delay(K) = f_delay(K-1);
end
f_delay(1) = f(J);
[y(J),b1] = LSadapt(f_delay(Delta+1),f(J));
e(J) = f(J) - y(J);
258
b(J,:) = b1;
end;
%
wavplay(e,Fs);
The script reads the sound le, adds the interference waveform and plays the
le. It then lters the le and plays the resulting output. After ltering the
sliding sinusoid can only be heard very faintly in the background. There is some
degradation in the quality of the speech, but it is still very intelligible.
This example was demonstrated in class at this time.
The following plot shows the waveform spectrum before ltering. The superpo
sition of the speech spectrum on the pedestal spectrum of the swept sinusoid can
be clearly seen.
Input Spectrum
2500
2000
1500
Magnitude
1000
500
0
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000
Frequency (Hz)
The pedestal has clearly been removed after ltering, as shown below.
Filtered Output Spectrum

1000
900
800
700
600
Magnitude
500
400
300
200
100
0
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000
Frequency (Hz)
259
The magnitude of the frequency response lter as a meshed surface plot, with
time as one axis and frequency as the other. The rejection notch is clearly visible,
and can be seen to move from a low frequency at the beginning of the message
to approximately 2.5 kHz at the end.
20
Magnitude (dB)
20
40
60
10
4000
6 3000
4 2000
2 1000
Time (sec) 0 0 Frequency (Hz)
Example 3
Adaptive System Identication:
An adaptive LMS lter may be used for real-time system identication, and will
track slowly varying system parameters. Consider the structure shown in below.
f(n ) u n k n o w n L T I s y s te m s y s te m o u tp u t
h (m )
d (n )
c a u s a l lin e a r F IR filte r y (n )
+
H (z ) -
e s tim a te d im p u ls e
re s p o n s e
filte r c o e ffic ie n ts h (m )
a d a p tiv e L e a s t- S q u a r e s e (n )
a lg o r ith m e rro r
2510
A linear system with an unknown impulse response is excited by wide-band

excitation f (n), The adaptive, length M FIR lter works in parallel with the
system, with the same input. Since it is an FIR lter, its impulse response is the
same as the lter coecients, that is
h(m) = b(m), for m = 0, 1, 2, . . . M 1.
and with the error e(n) dened as the dierence between the system and lter
outputs, the minimum MSE will occur when the lter mimics the system, at
which time the estimated system impulse h (m) response may be taken as the
converged lter coecients.
Consider a second-order unknown system with poles at z1 , z2 = Rej , that is
with transfer function
1
H(z) = ,
1 2Rcos()z 1 + R2 z 2
where the radial pole position R varies slowly with time. The following MATLAB
script uses LSadapt() to estimate the impulse response with 10,000 samples of
gaussian white noise as the input, while the poles migrate from z1 , z2 = 0.8ej/5
to 0.95ej/5
% Adaptive SysID
f = randn(1,10000);
% Initialize the filter with M = 2, Delta =.8
% Choose filter gain parameter Lambda = 0.1
Lambda = 0.01; M = 51;
x = LSadapt(initial,Lambda,M);
% Define the "unknown" system
R0 = .8; R1 = 0.95; ctheta = cos(pi/5);
delR = (R1-R0)/L;
L = length(f);
b=zeros(M,L);
ynminus2 = 0; ynminus1 = 0;
for J = 1:L
% Solve the difference equation to determine the system output
% at this iteration
R = R0 + delR*(J-1);
yn = 2*R*ctheta*ynminus1 - R^2*ynminus2 + f(J);
ynminus2 = ynminus1;
ynminus1 = y;
[yout,b(:,J)] = LSadapt(f(J),yn);
end;
The following plot shows the estimated impulse response, h (m) = b(m), as the
poles approach the unit circle during the course of the simulation, demonstrating
that the adaptive algorithm is able to follow the changing system dynamics.
2511
1.5
Impulse response h(n)
0.5
0.5
0.95
1
1.5 0.9
0
10 0.85 Pole radius
20
30
40 0.8
Time step (n) 50
1.2 The Recursive Least-Squares Filter Algorithm

The recursive-least-squares (RLS) FIR lter is an alternative to the LMS lter described
above, where the coecients are continually adjusted on a step-by-step basis during the
ltering operation. The lter structure is similar to the LMS lter but diers in the internal
algorithmic structure.
Like the LMS lter, the RLS lter is FIR of length M with coecients bk , k = 0, 1, 2, . . . , M
1. The input stream {f (n)} is passed through the lter to produce the sequence {y(n)}. At
each time-step the lter coecients are updated using an error e(n) = d(n) y(n) where
d(n) is the desired response (usually based of {f (n)}).
The LMS lter is implicitly designed around ensemble statistics, and uses a gradient
descent method based on expected values of the waveform statistics to seek optimal values
for the lter coecients. On the other hand, the RLS lter computes the temporal statistics
directly at each time-step to determine the optimal lter coecients. The RLS lter is
adaptive and can adjust to time varying input statistics. Under most conditions the RLS
lter will converge faster than a LMS lter.
Refer to the class handout Introduction to Recursive-Least-Squares (RLS) Adaptive Fil
ters for details.
2512

Signal Processing MIT PDF

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Signal Processing MIT PDF

Enviado por

Direitos autorais:

Formatos disponíveis

MIT OpenCourseWare

2.161 Signal Processing: Continuous and Discrete

Department of Mechanical Engineering

2.161 Signal Processing - Continuous and Discrete

Class handout: The Dirac Delta and Unit-Step Functions

1 Introduction to Signal Processing

Some typical applications that we look at will include

1.1 Processing methods

V in (t) C v o (t) V i n (t) C C v (t)

or even an electro mechanical system using rotational elements:

a sequence of samples {fn } where fn = f (nT ) (T is the sampling interval.

yn = (fn + fn1 + fn2 )

3-point moving average lter, and

2 Properties of LTI Continuous Filters

A LTI lter is dynamical SISO (single-input single-output) linear system, governed by an

y3 (t) = ay1 (t) + by2 (t).

3 The Dirac Delta Function

but with the requirement that

4 Properties of the Delta Function

4.0.2 The strength of an impulse

4.0.3 The Sifting Property of the Impulse

from the unit area property.

4.0.5 Laplace Transform

4.0.6 Fourier Transform

5 Practical Applications of the Dirac Delta Function

Y (s) = H(s)U (s).

y(t) = h(t) = L1 {H(s)} .

is used in sampling theory. A continuous waveform f (t) is sampled by multiplication

where f (t) is the sampled waveform, producing a train of weighted impulses.

2.161 Signal Processing: Continuous and Discrete

Department of Mechanical Engineering

2.161 Signal Processing - Continuous and Discrete

Class handout: Sinusoidal Frequency Response

1 Continuous LTI System Time-Domain Response

u (t) L T I F ilte r y (t) = ?

uT (t) = u(nT ) for nT t < (n + 1)T.

The approximation uT (t) is written as a superposition of non-overlapping pulses

u(nT ) nT t < (n + 1)T

where h(t) is dened to be the system impulse response,

h(t) = lim hT (t).

See the class handout for further details and examples.

2 Sinusoidal Response of LTI Continuous Systems

an (j)n + an1 (j)n1 + + a1 (j) + a0 Bejt

= bm (j)m + bn1 (j)m1 + + b1 (j) + b0 ejt

and solving for B

an (j)n + an1 (j)n1 + + a1 (j) + a0

H(j) is a property of the system. It is dened entirely by the describing dierential

H(j) = H(j), where H(j) is the complex conjugate.

u(t) = A sin (t + ) = e ej(t+)

y(t) = H(j)ej(t+) H(j)ej(t+)

y(t) = A |H(j)| sin (t + + H(j))

is described by the dierential equation

and is now described by the dierential equation

Clearly, as 0, |H(j)| 0, and H(j) /2 rad (90 ). As ,

2.161 Signal Processing: Continuous and Discrete

Department of Mechanical Engineering

2.161 Signal Processing - Continuous and Discrete

1 The Fourier Series and Transform

In this lecture we generalize the response of LTI systems to non-sinusoidal inputs. We do

x 1 (t) x (t) x 3 (t) T

The fundamental angular frequency 0 (in radians/second) of a periodic waveform is dened

See the class handout for details.

2.1 Computation of the Fourier Coecients