EECS 16B

Designing Information Devices and Systems II

Spring 2016

Official Lecture Notes

Note 1

Wireless communication and modeling signals

Wireless communication through radio waves is one of the foundational technologies for the contemporary
age directly responsible for hundreds of billions of dollars worth of economic activity in the world and
indirectly supporting trillions more. It is a critical enabler for other key technologies like satellites, deepspace exploration, drones, and robotics. Wireless is the key to the Internet Of Things as well as many
advances yet to come in personal health.
Wireless communication also happens to be a wonderful vehicle for learning how to take what we learned in
16A linear algebraically about inner-products, orthogonality, eigenspaces, and diagonalization to understand
powerful techniques for understanding signals in general as well as a very important category of linear
systems. So although this note is largely written with a wireless motivation, the ideas here are far more
general. In particular, you will see in the labs that the frequency-domain, introduced in this note, is useful
in interpreting recordings of neural data!

Signals and Systems

Were going to be talking about signals and systems that process those signals. At its heart, a signal is just a
function and therefore has a domain and a range. The voltage on a wire varying over time or the brightness
of a image varying as we scan across it are both examples of a signal. There are three broad classes of
signals:
Continuous-time signals Signals in the real world are analog in nature. Both the domain and the range of
the signal are both continuous. A continuous-time signal is defined as a function x : R 7 R (or C).
The following graph depicts a speech waveform represented as an analog signal over time:

Discrete-time signals When we store and process signals with computers, it is infeasible to read in the
entire continuous signal. Hence we usually take discrete samples of the signal to form a sequence. A
EECS 16B, Spring 2016, Note 1

discrete-time signal is defined as a function x : Z 7 R (or C). Because these signals are only defined
on discrete values in the domain (they are undefined at all other points), they, strictly speaking, cannot
be drawn as a continuous graph. The following graph shows the speech waveform from before, except
sampled into discrete-time.

Finite-time signals Especially when dealing with computers, it is often impossible to deal with a truly
infinite sequence. A finite-time signal is defined as a function x : Zn 7 R (or C) where Zn is the finite
set {0, 1, . . . , n 1}.
The graph above, and all graphs we can realistically plot on a piece of paper, is actually of a finite-time
signal since it does not extend forever.
A system is a function that takes in a signal and outputs another signal. Continuous-time real-world systems
such as analog electronics input and output continuous-time signals, whereas finite-time systems are things
like computer programs that manipulate stored values in arrays. These input and output finite-time signals.
Computer systems or digital electronics that keep on measuring the world in some manner and making
outputs can be thought of as naturally dealing with discrete-time signals. In this course we will start by
dealing with finite-time systems and signals because these can be modeled as matrices and vectors. Later
on, as we cover differential equations and control, we will start thinking in continous time too.

Linear Time-Invariant Systems and Echoes in Wireless Channels

It turns out that a very important category of systems are linear time-invariant (LTI) systems. Perhaps
the simplest physical example of an LTI system is provided by looking at what nature does to wireless
transmissions.
In the real-world, the radio waves emitted by a wireless transmitter dont only go directly to the wireless
receiver that wants to listen to them. Instead, they propagate physically, much the same way that sound
propagates. If there is a clear line-of-sight path between the transmitter and the receiver, then certainly the
radio waves do reach the destination directly. But they also find other paths, bouncing off of obstacles like
walls, the ground, the ceiling, buildings, hills, etc. Because the speed of light is finite1 , all of these echoes
of the transmitted signal reach the destination with different delays2
1 The speed of light is easily remembered as 3 108 m/sec, which is 300 meters per microsecond or 15 meters per 0.05 microseconds which is the sampling rate of many WiFi implementations 20 samples per microsecond.
2 Paths that are longer by a full 15 meters (say from a wall across the room) show up as a full sample later, while less than that,
they for practical reasons show up as a pair of echoes one at zero samples and another at 1 sample. More details about this are
discussed later in 16B.

EECS 16B, Spring 2016, Note 1

The figure below3 shows an example of what real-world radio echoes look like:

The vertical scale in this plot is logarithmic (dB scale is 10 log10 ()) because the echoes tend to decay
exponentially in amplitude as they encounter more and more bounces. The scales are normalized so that
the peak is always at 0 in log-scale = 1 in linear scale. The no-line-of-sight plot on the right shows that the
echoes can be quite significant for over 200nsec which corresponds to about 4 samples in a system that uses
20MHz wide channels like 802.11. The term impulse response is just used to mean what the response of
the channel would look like to a hypothetical transmission of a single pulse. If there were no obstacles and
we were in free-space, the response would just look like a single peak going up to 0dB about 17 nsec later
since it takes light 17nsec to travel 5 meters.
It is useful to caricature the response of a wireless channel as just having a few paths. For example, if there
were three paths, one with delay 1 , another with delay 2 > 1 , and a third with delay 3 > 2 > 1 , then
the received signal y(t) = 1 x(t 1 ) + 2 x(t 2 ) + 3 x(t 3 ) in continous time. No matter what is sent,
the receiver would hear the superposition of three different copies of the transmitted signal, each shifted by
delay i and weighted by gain i .
This kind of response is called linear and time-invariant for very natural reasons. It is a common feature in
natural and engineered systems, not just the wireless communication channel.

Linearity
Given a potentially complicated signal, how do we determine how a system acts on it? As we have seen in
circuits, we can break large networks into simpler parts, compute the results for each of the parts, and sum
the results. This general principle of breaking a complex system into smaller parts, performing an operation
on each part and combining the results to get the result of performing the operation on the entire system, is
called superposition, and we would like to apply it to signals and systems too. Firstly, what kind of systems
allow us to apply the principle of superposition? Suppose we have a system F, a function taking in signals.
Now if we have a signal z = x + y, we want the output of F on z equal to the sum of the outputs of F on x
3 Taken/Photoshopped

from: Hashemi, Homayoun. "The indoor radio propagation channel." Proceedings of the IEEE 81.7

(1993): 943-968.

EECS 16B, Spring 2016, Note 1

and y individually. More precisely:

F(z) = F(x + y) = F(x) + F(y)

(1)

Also if we scale the input signal, we want the output signal to scale by the same amount:
F(cx) = cF(x)

(2)

This is exactly the same properties that linear transforms in linear algebra have, in fact linearity is a very
important concept that arises in many different areas of mathematics and the sciences.
Notice that it holds for the wireless channel with echoes. If we increase the amplitude of the transmitted
signal, the received signal increases by the same factor. If we add together two signals and send them, the
response is the same as if we had sent them individually and then added the result. Mathematically, if the
transmitted wavefom x(t) = xa (t) + xb (t), the received waveform is:
y(t) = 1 x(t 1 ) + 2 x(t 2 ) + 3 x(t 3 )

(3)

= 1 ( xa (t 1 ) + xb (t 1 )) + 2 ( xa (t 2 ) + xb (t 2 )) + 3 ( xa (t 3 ) + xb (t 3 )) (4)

= (1 xa (t 1 ) + 2 xa (t 2 ) + 3 xa (t 3 )) + (1 xb (t 1 ) + 2 xb (t 2 ) + 3 xb (t 3 )) (5)

= ya (t) + yb (t)

(6)

where ya is the response to just xa and yb is similarly the response to just xb .

Time-Invariance
Many physical systems dont care about what the absolute time is. For such systems, if an input signal
is shifted in time, the output signal should also be the same, except it is also shifted in time by the same
amount:
F(x(n + s)) = F(x)(n + s)

(7)

Notice that this also holds4 for the wireless echo channel. It only cares about the relative delays, not absolute
time. To verify just notice:
y(t + s) = 1 x(t + s 1 ) + 2 x(t + s 2 ) + 3 x(t + s 3 )

(8)

4 Approximately. In real life, things move around and so this sort of thing holds for short enough delays but not long ones
where the physical paths themselves have changed. Fortunately for us, in the real world, reflecting objects and transmitters and
receivers all move very slowly relative to the speed of light and the sampling periods of modern wireless systems. So the channel
is approximately LTI for at least a couple hundred microseconds or so.

EECS 16B, Spring 2016, Note 1

and so it is time-invariant by inspection. A shift of the input shifts the output.

Linear Algebraic Representation

By now, we have seen that a linear system shares a lot in common with linear transforms on vectorspaces.

x0
x1

Let us view signals and systems in the language of linear algebra. Suppose we have a signal ~x = .
.
.
x

n1
y0
y1

represented by a vector, and a system S. The system matrix acts on x to give the output5 ~y = . :
.
.
yn1

x0
y0
. .
S
.. = ..
xn1
yn1

(9)

We already know that multiplication of a vector by a matrix is a linear transform, so S is a linear system.
How about time-invariance? First we need to figure out what happens to the signal when we apply a timeshift to it. Suppose we want to shift our signal ~x by 3 steps. Now we have ~x3 = [_, _, _, x0 , x1 , . . . , xn4 ]T .
Notice that we have 3 spaces at the beginning of the signal that need to be filled in. Also, xn3 , xn2 and
xn1 are missing.
To preserve symmetry and to avoid losing information, its best to assume that the original signal is periodic,
and ~x and ~x3 are both one period of the original signal, except starting at different times. Therefore
~x3 = [xn3 , xn2 , xn1 , x0 , x1 , . . . , xn4 ]T . More formally, we have:
~xk = [x0k , x1k , . . . , xn1k ]T

(10)

Where all the indices additions are done modulo6 n .

This twist is required to define time-invariant systems for finite-time. It views times as being arranged in a
circle like the face of a clock. So shifting by 1 is just rotating the circle by 1 position.
Exercise 1.1 (Time-Shift Systems): Imagine we have a system S2 that takes any length 5 input signal
and shifts it by 2 steps. For example, S2 ([3, 1, 4, 1, 5]) = [1, 5, 3, 1, 4].
1. Is this system linear? That is, for any signals ~x and ~y, does S2 fulfill properties (1) and (2)?
2. Is this system time-invariant? Does it fulfil (7)?
5 Note

on notation:
  notice that writing vectors vertically can be quite space-consuming and cumbersome. So from now on
a
instead of using
, we will use the equivalent, less vertical-space consuming [a, b]T .
b
6 a modulo b (or a (mod b)) is the same as adding or subtracting multiples of b to a until it is between 0 and b 1. For example,
12 (mod 5) = 2 and 2 (mod 3) = 1.
EECS 16B, Spring 2016, Note 1

3. What does S2 look like when written as a matrix?

Do this exercise before reading on.
Now that we have defined a time-shift, lets look at the structure of a system that is time-invariant. This
means that S (~xk ) = (S~x)k . More concretely,

..
..

. .

xn1 yn1

x0 = y0
S
(11)

x1 y1

..
..
.
.
This still doesnt seem to tell us too much about the structure of S. To see that structure, remember that
linearity allows us to break a signal into smaller constituent pieces. First, just look at the first component of
~x, [x0 , 0, . . . , 0]T , and how the system transforms it:

s0
s0
s1
x0

0
s1
s2

(12)

.. = x0 ..
..
.
.
.

0
sn1
sn1
We can see that this input signal picks out the first column of the signal matrix. What if we shift the signal
by one time-step? Since this system is time-invariant, we must have:

0
sn1

s0

x0

. = x0
S
(13)
..

.
.

.
0
sn2
Hence we can deduce that the second column of S is [sn1 , s0 , . . . , sn2 ]T ! In fact, the entire matrix can be
deduced by shifting the input signal n times:

s0 sn1 sn2
s1
s1
s0 sn1
s2

s1
s0 s3
S = s2
(14)

..
..
..
..
.
.
.
.
sn1 sn2 sn3
s0
Therefore, all finite-time linear time-invariant (LTI) systems can be described by matrices of the form above.
We have seen such matrices before when we were defining correlations between signals in the context
of estimating time-delays for locationing. They are called circulant matrices. We used the notation C~s to
describe making such a matrix from its first column ~s.
So how many parameters do we need to uniquely determine the system? The distinct parameters making
up the matrix S are s0 , . . . , sn1 , so we only need n parameters. Suppose we have a black box system and
we want to find out these parameters; we know that it is LTI, and we can input any signal and observe the
EECS 16B, Spring 2016, Note 1

output. What signal should we input? You might recall from above that if we just input ~ = [1, 0, . . . , 0]T , we
will get ~s = [s0 , s1 , . . . , sn1 ]T , all the parameters we need! The signal ~ is called an impulse, and the output
~s of the system is called the impulse response. Hence we see that an LTI system is completely determined
by its impulse response.
All this can be generalized to the case of continuous-time as well as discrete-time that is infinite in extent,
but the mathematics becomes trickier. We will deal with some of this later in 16B and you will see more in
120, but for now, we will stick largely to finite-time.

A Natural Basis For LTI Systems: the DFT Basis

We have seen that an LTI system that acts on a signal of length n can be completely determined by n pieces
of information, s0 , s1 , . . . , sn1 . However, it is cumbersome to do an entire matrix multiplication to compute
the output of a system on signal ~x = [x[0], . . . , x[n 1]]T . Can we transform ~x and S to a different basis, so
that we get ~x0 = [x0 [0], . . . , x0 [n 1]]T and S0 is a diagonal matrix? If this is possible, then:
0
0

s0
x [0]
s00 x0 [0]

x0 [1] s0 x0 [1]
s01
1

S0~x0 =
(15)

..
..
..

.
.
.
s0n1

x0 [n 1]

s0n1 x0 [n 1]

And we have transformed a matrix multiplication into a simple component-wise multiplication of one vector
with another. This is possible only if we can diagonalize S. If we can write S = US0U 1 and ~x = U~x0 , where
U is the change-of-basis matrix:
S~x = (US0U 1 )(U~x0 ) = U(S0~x0 )

(16)

Therefore if we work in the basis defined by the column vectors of U, S will become a diagonal matrix and
simplify a lot of operations.
How do we find this special basis? Recall that to diagonalize a dimension n matrix, we need to find n linearly
independent eigenvectors. Then U is the matrix with the eigenvectors in its columns and S0 is a matrix with
the corresponding eigenvalues on its diagonal. Therefore we need to find the eigenvectors and eigenvalues
of S.
In general, this matrix can be very large and so the usual methods for finding eigenvalues by finding the
roots of its characteristic polynomial (setting det(S I) = 0 and solving for the roots ) would give us no
insight. However we will exploit the special symmetry inherent in S. Lets start by examining n = 3 so we
dont get bogged down with generality:

s0 s2 s1
S = s1 s0 s2
(17)
s2 s1 s0
Let ~v = [w0 , w1 , w2 ]T . Then

s0 w0 + s1 w2 + s2 w1
S~v = s0 w1 + s1 w0 + s2 w2
s0 w2 + s1 w1 + s2 w0
EECS 16B, Spring 2016, Note 1

(18)

For ~v to be an eigenvector, S~v = ~v. What properties should w0 , w1 and w2 have so that ~v is an eigenvector?
We see an intriguing pattern in S~v. If wi has the following two properties:
1. wi w j = wi+ j

(mod n)

2. w0 = 1
Then we can factor out the eigenvalue:

w0
(s0 w0 + s1 w2 + s2 w1 )w0
(s0 w0 + s1 w2 + s2 w1 )w0 w0
S~v = (s0 w1 + s1 w0 + s2 w2 )w2 w1 = (s0 w0 + s1 w2 + s2 w1 )w1 = s0 w0 + s1 w2 + s2 w1 w1
w2
(s0 w0 + s1 w2 + s2 w1 )w2
(s0 w2 + s1 w1 + s2 w0 )w1 w2

(19)

Therefore ~v is an eigenvector with eigenvalue s0 w0 + s1 w2 + s2 w1 .

This guessing and checking method we used might seem quite magical, but it is often a great way to find
eigenvectors of matrices with a lot of symmetry.
To really find the eigenvectors, we need to find a set of numbers w0 , w1 , w2 with the above-mentioned
properties. Real numbers wouldnt do the trick, as they dont have the wrap-around property needed. For
2m
all m = 0, 1, . . . , n 1, complex exponentials of the form wk = ei n k have all the required properties.
2

Why? Because ei n k steps along the n-th roots of unity as we let k vary from 0, 1, . . . , n 1. This is a simple
2
2
consequence of Eulers identity ei = cos + i sin geometrically or the fact that (ei n )n = ei n n = ei2 = 1.
How could we have come up with this guess? The 3x3 case is not impossible to solve. We couldve just
written out the matrix for s0 = s2 = 0 and s1 = 1. (This is the shift-by-one matrix that simply cyclically
delays the signal it acts on by 1 time step.) At this point, you could explicitly evaluate the det(S I) to
find the characteristic polynomial and see that we need to find the roots of 3 1. This implies that the
eigenvalues are the 3rd roots of unity. From there, you can find the eigenvectors by direct computation and
you would find:
The 3 eigenvectors are:

1
v0 = 1 ,
1

2i
v1 = e 3 ,
4i
e3

4i
v2 = e 3
2i
e3

(20)

You would then notice that the same eigenvectors work for the case s0 = s1 = 0 and s2 = 1. This is the delay
by 2 matrix which, by inspection, can be seen as being D2 if D is the delay-by-one matrix. At this point, you
know that since these every 3x3 circulant matrix is a linear combination of the identity matrix (no delay)
and these two pure delay matrices, and all three of these matrices share the same eigenvectors, that in fact
these must be the eigenvectors for all 3x3 circulant matrices.
Generalizing, if we have a n-dimensional signals, all LTI systems that act on such signals would have the
same n eigenvectors:
~vl = [ l0 , l1 , . . . , l(n1) ]T

EECS 16B, Spring 2016, Note 1

(21)

Where = e
be 1.

2i
n

. To normalize this, we have ~ui =

1 ~
v.
n i

As we will see, the extra

1
n

term makes the norm

Therefore we can write U as:

U = [~u0 ~u1

1
1
1

1
.
..
~un1 ] = ..
.
n
1 n2
1 n1

..
.

n2
..
.

n1
..
.

(n2)(n2)
(n2)(n1)

(n1)(n2)

(n1)(n1)

(22)

The basis in which S is diagonalized is called the Fourier basis or DFT basis (also called the frequency
domain), and U as a linear transformation is called the inverse discrete Fourier transform that brings
signals from the frequency domain into the time domain. The conjugate transpose of U, namely U , maps
signals from the time-domain to the frequency domain and as a transformation is called the discrete Fourier
transform or DFT.
We have seen from above that the Fourier basis is a natural basis when we are dealing with LTI systems, and
the Fourier transform is an important bridge between the time and frequency domains.
For the wireless communication context, it means that no matter what pattern of echoes we encounter, the
DFT basis vectors ~uk are eigenvectors. This means that we can safely encode messages to different receivers
onto the (complex) amplitudes of different DFT basis vectors and we are guaranteed that they will not
interfere with each other. Each receiver can get their message back without experiencing any interference
from the messsages being communicated to other receivers. All they have to deal with is the eigenvalue
of the matrix that hits their particular ~uk . In the wireless communication context, the fact that the DFT
basis is the universal basis for all LTI systems means that the transmitter and receiver can agree on the
basis without having learned the details of the specific wireless channel (pattern of echoes) that they are
experiencing. This is very different from what would happen if messages were attempted to be separated
using the standard basis. The wireless channel would induce interference between the messages.

Some Properties of the DFT Basis

Illustrating the DFT Basis Vectors
It is worth actually looking at an example of what the DFT vectors look like. These are complex exponentials, and as the frequency k increases, they wiggle more and more. They always have an integer number
of periods between in the total duration n.
k

uk [t]

Unit circle plots

EECS 16B, Spring 2016, Note 1

Cartesian coordinates plots

Polar coordinates plots

=
u[3]
u[4]

u[2]
u[1]

u[5]

u[0]

u[6]

u[9]
u[7]

2.i.1.t
10

u[8]

=
u[7]
u[2]

u[6]
u[1]
u[0] u[5]
<

u[3]
u[8]

<

u[4]
u[9]

2.i.2.t
10

=
u[8]

u[1]

u[5]

u[0]
u[2]

2.i.3.t
10

u[7]

u[4]

u[9]

EECS 16B, Spring 2016, Note 1

u[3]

<

u[6]

10

=
u[6]
u[1]

u[3]
u[0] u[5]
<

u[4]
u[9]

u[8]

u[2]
u[7]

2.i.4.t
10

u[5] u[3]
u[7]
u[9]
u[1]

u[0]
u[2] u[8]
u[4]
u[6]
<

2.i.5.t
10

=
u[9]
u[4]

u[1]
u[6]

2.i.6.t
10

EECS 16B, Spring 2016, Note 1

u[7]
u[2]
u[0] u[5]
<
u[3]
u[8]

11

u[9]

u[6]
u[3]

u[2]

u[0]

u[5]
u[8]

u[1]

u[4]

u[9]

u[4]
u[0] u[5]
<

u[2]
u[7]

u[1]
u[6]

2.i.8.t
10

u[7]

u[8]
u[9]

u[6]
u[5]

O
u[4]
u[3]

u[7]

2.i.7.t
10

u[8]
u[3]

<

u[0]

u[1]
u[2]

<

2.i.9.t
10

EECS 16B, Spring 2016, Note 1

12

Orthonormality of the DFT Basis

The fact that the DFT Basis U is orthonormal lets us use the conjugate transpose U as its inverse and this
makes it easy to go back and forth between time domain (standard basis) and frequency domain (DFT basis).
The following diagram illustrates how one goes back and forth.

Time-Domain
(Standard Basis)

Frequency-Domain
(DFT Basis)

Signals

U ()
~s : s[t]

U()

~S : S[ f ]

To prove orthonormality, we have to show two things:

The DFT basis vectors each have norm one.
k~uk k2 = h~uk ,~uk i
n1

t=0
n1

n ei

t=0
n1

( n ei

2
n kt

2
n kt

1 2
) ei n kt
n
2

ei n kt

t=0

= 1.

EECS 16B, Spring 2016, Note 1

13

 The DFT basis vectors are orthogonal to each other. So if k 6= `

n1

h~uk ,~u` i =
=

n ei

t=0
n1

n ei

2
n kt

ei n `t

2
n (`k)t

t=0
n1

2
1
(ei n (`k) )t

n t=0
2

1 (ei n (`k) )n 1
= ( 2
)
n (ei n (`k) ) 1
1 (ei2(`k) ) 1
= ( 2
)
n (ei n (`k) ) 1

1
11
)
= ( 2
n (ei n (`k) ) 1
=0

The above is an argument by direct computation using the formula for the sum of a geometric series.
It can also be seen more intuitively by plotting the points in the sum on the complex plane and seeing
how they are balanced around 0.
It is useful to see some examples of what different signals look like in both time-domain and frequency
domain. Here are some examples:
Example

Time domain

Frequency domain

Constant signal

(imaginary part)

Spike signal

EECS 16B, Spring 2016, Note 1

14

(imaginary part)

Shifted spike signal

(imaginary part)

Complex exponential

(imaginary part)

The DFT Basis diagonalizes all LTI Systems

This is perhaps the most important property of the DFT basis: the DFT vectors are eigenvectors for all
finite-time LTI systems, in other words, for all circulant matrices.
This result, which we have already argued informally is true, is something that we will establish more
formally below. But first, it is useful to see a diagram that points out how LTI Systems look in the time and
frequency domain and how these two are connected.

EECS 16B, Spring 2016, Note 1

15

Time-Domain
(Standard Basis)

Systems

h[0]
h[1]
..
.

h[n 1]

h[n 1]
h[0]
..
.

...
...
..
.

h[n 2]

...

Frequency-Domain
(DFT Basis)

h[1]
H,0

U()U
0
h[2]

H = .
.. = H
..
.
U ()U
h[0]
0

H,1
..
.

...
...
..
.

0
0
..
.

...

H,n1

To justify the above diagram, we just need to why the DFT vectors are eigenvectors of all circulant matrices.
We have already established (by showing orthonormality) that the DFT basis has dimension n and that
U 1 = U . Once we know this, the above diagram follows from what we already know about diagonalization
of matrices.
We will proceed by first considering special cases of circulant matrices. learly this is true for the Identity
matrix since everything is an eigenvector of the identity. Now consider the delay by 1 matrix D. This has
impulse response d~ = [0, 1, 0, . . . , 0]T and D = Cd~ is the circulant matrix that has d~ as its first column.
Lets see what it does to the DFT basis elements.
1
(D~uk )[t] = vk [(t 1) mod n]
n
1 2
= ei n k((t1) mod n)
n
1 i 2 k((t1)qn)
= e n
n
1 i 2 k(t1) i 2 kqn
e n
= e n
n
1 2
= ei n k(t1) ei2kq
n
1 2
= ei n k(t1) (ei2 )kq
n
1 2
= ei n k(t1)
n
2
1 2
= (eik n ) ei n kt
n

(23)
(24)
(25)
(26)
(27)
(28)
(29)
(30)

= (eik n )uk [t]

(31)

The third line in the derivation above can be justified since we know that (t 1) mod n = t 1 qn for some
integer q since mod means remainder after dividing by n.
2

So, indeed the matrix D has ~uk as an eigenvector and has eigenvalue D,k = eik n .
Next, observe that a cyclic delay by two can be obtained by first delaying by 1 and then doing it again. In
other words D2 is a delay by two. By this exact reasoning, notice that a cyclic delay by {1, . . . , n 1} is
EECS 16B, Spring 2016, Note 1

16

just D and by diagonalization, it is clear that D has ~uk as an eigenvector too and the eigenvalue is also just
= eik 2
n .
raised to the power . So D ,k = D,k
Now, we know that the DFT basis are eigenvectors for D for {0, 1, . . . , n 1}. This means that the
DFT basis must also be eigenvectors for all linear combinations of powers of D. This can be seen formally.

u` .
Consider the matrix H = n1
=0 h[]D acting on a basis vector ~
n1

H~u` = ( h[]D )~u`

(32)

=0
n1

h[]D~u`
=0
n1

h[]ei`

(33)
2
n

=0
n1

= ~u`

~u`

(34)

2
n

(35)

h[]ei`

=0
n1

= ( h[]ei` n )~u`

(36)

=0

This shows that indeed ~u` is an eigenvector and has

n1

H,` =

h[]ei`

2
n

(37)

=0

as the corresponding eigenvalue.

Finally, notice that by choosing the vector ~h = [h[0], h[1], . . . , h[n 1]]T arbitrarily, the above matrix H is
just a general circulant matrix! This can be seen most clearly by just looking at the first column. The first
column of D has a 1 in the position and zeros everywhere else.
We have now established that the DFT basis is an eigenvector basis for all the circulant matrices.

In the DFT Basis, an LTI system just scales coefficients

The fact that the DFT Basis diagonalizes all LTI systems means that we can very easily compute the coordinates in the DFT basis for what happens when a general finite-time signal7 passes through an LTI system.
Given an input signal ~s, suppose that its representation in DFT basis was ~S so that ~s = n1
u f . Then,
f =0 S[ f ]~
n1
n1
~
if the LTI System (Circulant Matrix) H acts on s, we know that H~s = f =0 S[ f ]H~u f = f =0 S[ f ]H, f ~u f .
So the coordinates in the frequency domain are simply multiplied by the eigenvalues H, f to compute the
coordinates of the resulting signal in the DFT basis.
This is illustrated in the example below.
7 Recall

that physically, such finite-time signals are understood to be periodic in the real world. This is what makes the shifts be

cyclic.
EECS 16B, Spring 2016, Note 1

17

Time domain

Freq domain (magnitude)

Freq domain (phase)

Input signal

Impulse response

Output signal

Notice here how the polar coordinate representation of the frequency domain makes it easier to see how the
output is just the input multiplied by the eigenvalues of the system matrix. This is because the magnitudes
multiply and the phases just add when multiplying two vectors by each other component-wise.
We notice another very important fact in this example plot when we consider the relationship between the
frequency components of the impulse response and the eigenvalues of the circulant matrix that has this
impulse response. This is the topic of the next subsection of this note.

The eigenvalues of an LTI System are scaled versions of the frequency components of the impulse
response when viewed in the DFT Basis
There is a close relationship between the representation of a finite signal ~h in frequency domain and the
eigenvalues of the circulant matrix C~h representing the system whose impulse response is ~h.
~ = U ~h is the representation of ~h
Because the DFT basis U is orthonormal, we know that U 1 = U . So H

EECS 16B, Spring 2016, Note 1

18

~
in the frequency domain. Consider the f -th component of H:
H[ f ] = (U ~h)[ f ]
n1

(38)

h[] n (ei f

2
n

(39)

=0

2
1 n1
= h[]ei f n
n =0
1
= C~h , f
n

(40)
(41)

Where the last line was obtained by comparing this to (37).

~ refers to ~h viewed in the DFT basis while H = C~ is the
This correspondance justifies the notation where H
h
circulant matrix made from the impulse response ~h.
This can also be seen from a diagonalization perspective
C~h = UU
U C~h = U UU
= U
Now just look at the first column of both sides, in other words look at the standard basis vector ~e0 postmultiplying both sides.

U C~h~e0 = U ~e0

1

1 1

U ~h = .
.
n .
1

C~h ,0

1 C~ ,1
U ~h = .h
n ..

C~h ,n1
Here, what turned out to be most important is that the first column of U is just
and that the U matrix is orthonormal and diagonalizes H = C~h .

1
n

repeated in every position

The following diagram shows abstractly what is going on. On the bottom, we have the LTI systems that
correspond to the signals on top. On the left, things are expressed in time-domain (with coordinates in the
the standard basis). On the right, things are expressed in frequency-domain (using the DFT basis). Moving
right to left and vice-versa are change of coordinates. Moving up to down takes a signal and views this as
the impulse response of the desired LTI system. Moving down to up takes an LTI system and gives you the
impulse response.

EECS 16B, Spring 2016, Note 1

19

Signals

Time-Domain
(Standard Basis)

~h : h[t]
()~e0

Systems

h[0]
h[1]
..
.

h[n 1]

U ()

Frequency-Domain
(DFT Basis)

~ : H[ f ]
H

U()
C()

h[n 1]
h[0]
..
.

...
...
..
.

h[n 2]

...

nDiagonal()

1 ()~1
n

h[1]
H,0

U()U

h[2]
0
H = .
.. = H
..
.
U ()U
h[0]
0

H,1
..
.

...
...
..
.

0
0
..
.

...

H,n1

1
1
0
1


In the above diagram, ~e0 = . is the standard basis vector with a 1 in the zeroth position, ~1 = . is the
..
..
0
1
vector with all 1s in it, diagonal() takes a vector and returns a matrix with the elements of that vector along
the diagonal and zeros everywhere else, and C() takes a vector and returns the circulant matrix with the first
column being that vector. U is the orthonormal DFT basis and U is its conjugate transpose.
The numpy.fft.fft function defaults to computing the eigenvalues given an impulse response. So
that corresponds to a move from the upper left to the bottom right.
The homework had an extended problem on OFDM modulation this shows how these DFT properties
turn out to be useful in the context of wireless communication. Not only can different users have their signals
encoded onto different complex exponentials, but we can exploit the nature of the DFT to even figure out
what the response of the wireless channel is even as we are communicating.

EECS 16B, Spring 2016, Note 1

20