Spectrum Estimation

Sherif H. El-Gohary , Phd

sherigoal@gmail.com

1
Lecture Outlines
•Introduction
Fourier Series and Transform
Time/Frequency Resolutions
Autocorrelation & spectrum estimation
•Non-parametric Methods
Periodogram
Modified Periodogram
Bartlett’s Method
Welch’s Method
Blackman-Tukey Method
•Parametric Methods

2
Fourier Series and Transform
Im

sin(k0t ) cos(k0t )
jk0t
e
Re
t

Fourier basis functions:

real and imaginar parts of a complex sinusoid
vector representation of a complex exponential.
3
 Fourier Series:

 ck e jk0t 1 T0 2

 jk0t
x(t )  , ck  x(t )e dt
k   T0 T0 2
k=…,-1,0,1,…

x(t ) c(k )
T0 n

k
T0 ff t 1
T0  Ton  Toff T0 4
 Fourier Transform:

x(t )   X ( f )e j 2ft 
df X ( f )   x(t )e  j 2ft dt
 , 

x(t ) X( f )

T0 ff 
t k

5
Discrete Fourier Transform (DFT)

x(0) X (0)
DFT
x(1) X (1)
N 1 2km
j
X (k )   x(m)e N

m 0

x( N  1) X ( N  1)

N 1 2km
x(m)   X (k )e
j
N
, k  ....,1,0,1,....
m 0

6
Introduction
• Spectrum Estimation: To estimate the
power spectral density (PSD) of signal.
• PSD is the DTFT of the autocorrelation
sequence.
• Estimating the power spectrum is
equivalent to estimating the
autocorrelation.
 Autocorrelation & Spectrum estimation

Autocorrelation:
N
1
lim { 
N  2 N  1 n   N
x(n  k ) x * (n)}  rx (k )

Power spectrum :

Px (e  jk
) x
r ( k
k  
) e  jk

Spectrum estimation is a problem that involves

estimating Px (e jt ) from finite number of noisy
measurements of x(n).
8
 Introduction
• Spectrum estimation is the problem that
involves estimating Px(ejw) from a finite number
of noisy measurement of x[n].
• Spectrum estimation is a problem that is
important in a variety of different fields and
applications.
• Applications: Optimum Filter design, Signal
Detection & Tracking, Harmonic Analysis &
Prediction, Spectral Smoothing, Beamforming &
Direction Finding,
 Techniques for Spectrum
Estimation
• Classical or Non-parametric Approach
– Estimating the autocorrelation sequence from
given set of data.
– Power Spectrum is then obtained by Fourier
transforming the estimated autocorrelation.
• Non-Classical or Parametric Approach
– Based on using a model for the process to
estimate the power spectrum.
– AR, MA or ARMA Models
Nonparametric methods

•Peroidogram
•Modified periodogram
•Bartlett method
•Welch method
•Blackman-Tukey method

11
The periodogram

Estimated autocorrelation:
N 1 k
1
ˆrx (k ) 
N
 x ( n  k ) x * ( n)
n 0

Estimated power spectrum or periodogram:


ˆ  jk
Pper (e )   rx (k )e
ˆ
k  
 jk

12
 The Periodogram
• If x[n] is only measured over a finite interval, say n=0,1,….,N-1 ,
then we can estimate the autocorrelation sequence using:

• In order to ensure that the values of x[n] that fall outside the
interval [0,N-1] are excluded, we can write:

k  0, rˆx (k )  rˆ* x (k )

| k | N rˆx (k )  0
 The Periodogram
• Taking the Fourier transform of the autocorrelation
sequence gave us the Periodogram.

• Next we will represent the Periodogram directly in

terms of the process x[n].
• Let xN[n] be the finite length signal of length ‘N’ that is
equal to x[n] over the interval [0,N-1] and is zero
otherwise:
 The periodogram cont.

x(n)

xN (n)  wR (n) x(n)

1  1
r̂x (k)   x N (n  k)x N (n)  x N (k)  x  N (k)


N n  N

ˆ j 1 j  j 1 j 2
Pper (e )  X N (e ) X N (e )  | X N (e ) |
N N
15
Example : Periodogram of White Noise

• If x[n] is the white noise with variance σx2

rx (k )    (k )
2
x

Px (e )  
jw 2
x
• Thus the power spectrum should be constant.
• Consider a sample realization of white noise of
unite variance and N=32
 The periodogram of white noise

x(n) : white noise with a variance  2, length N=32

1 1
xN (n) DFT X N (k ) | . |2 Pˆper (e j 2k N
) | X N ( k ) |2
N N

17
The periodogram of white noise cont.

The estimated autocorrelation sequence

18
White noise power spectrum
 Periodogram of sinusoid in noise

x(n)  A sin(n0   )  v(n)

1 2
A
2 j 1 2
Px (e )   v  A  (  0 )
2

 v2 2


0 19
 Periodogram of sinusoid in noise cont.

y1 (n) z1 (n)
win (1)
1
| . |2 ˆ j1
PX (e )  z1 ( N  1)
N

y 2 ( n) z 2 ( n)
PˆX (e j2 )  z2 ( N  1)
1
win (2) | . |2
N
x(n)

yL (n) z L (n)
win (L) 1
| . |2 ˆ jL
PX (e )  zL ( N  1)
N

20
 Matlab Code
clear all;
close all;
A=1;
N=100;
fs=0.3;

n=1:N;
rn=randn(1,N);
phi=2*pi*rand(1,N);
x=A.*exp(j*2*pi*fs*n+phi)+rn;
Px=abs(fft(x,1024)).^2/length(n);
w=2.*pi.*[0:1023]./1024;
plot(w./(2*pi),Px)
xlabel('\bf Normalized Frequency')
ylabel('\bf Periodogram')
Title('\bf [A=1,N=100]')

Matlab Function can also be used directly: periodogram ( )

 A=1
5
x 10 [A=1,N=100]
6 6
8 x 10 [A=1,N=1000] x 10 [A=1,N=10000]
7 7

7
6 6

6
5 5
5
Periodogram

Periodogram

Periodogram
4
4
4
3
3
3
2
2
2

1
1 1

0
0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized Frequency 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Normalized Frequency Normalized Frequency

N=100 N=1000 N=10000

[A=0.001,N=100]
7 [A=0.001,N=1000] [A=0.001,N=10000]
10 0.7

6 9
0.6
8
5 0.5
7

Periodogram
Periodogram

Periodogram

4 6 0.4

5
0.3
3
4
0.2
2 3

2 0.1
1
1
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized Frequency
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Normalized Frequency Normalized Frequency

A=0.001
Periodogram Bias
1 N 1 k
N k
rˆx (k ) E{}
N

n 0
rx (k ) 
N
rx (k )

  N | k |
 | k | 0
E{Pˆper (e j )}   rx (k )wB (k )e _ jk w B (k)   N
 0 o.w
1
E{Pˆper (e j )}  Px (e j ) WB (e j )
2

lim E{Pˆper (e j )}  Px (e j )
N 

Thus, the bias is deference between estimated and actual

Power spectrum. 23
 Periodogram of sinusoid in noise cont.

x(n)  A sin(n0   )  v(n)

1 2
A
4

ˆ 2 1 2
E{Pper (e )}   v  A [WB (e j ( 0 ) )  WB (e j ( 0 ) )]
j

2

 k  0 
2
k 0 24 
N
Example: x(n)  1sin(0.4n  )  v(n)

N  128 N  512
25
 Periodogram Resolution
x(n)  A1 sin(n1  1 )  A1 sin(n2  2 )  v(n)
j 1 2 1
Px (e )   v  A1  (  1 )  A2  (  2 )
2 2

2 2
ˆ 1 2
E{Pper (e )}   v  A1 [WB (e j ( 1 ) )  WB (e j ( 1 ) )]
j 2

4
1 2
 A2 [WB (e j ( 2 ) )  WB (e j ( 2 ) )]
4
Set  equal to the width of main lobe of the spectral window
at it’s half power or 6dB point.
ˆ j 2
Res[ Pper (e )]  0.89
N 26
Example: x(n)  1sin(0.4n  1 )  1sin(0.45n  2 )  v(n)

N  128 N  512
27
Properties of the periodogram
n 1 2
1
ˆ j
Pper (e ) 
N
 x
n 0
( n ) wR ( n ) e  jn

1
Bias: E{Pˆper (e j )}  Px (e j ) WB (e j )
2

2
Resolution: Res[ Pˆper (e j )]    0.89
N

Variance: Var{Pˆper (e j )}  Px2 (e j )

28
 Modified Periodogram
 2
1 1
Pˆper (e j )  | X N (e j ) |2 
N N
 x ( n) w
n  
R (n)e  jn

Would there be any benefit in replacing the rectangular window

with other windows? (for example triangular window)

ˆ j 1 j j 2 sin( N 2)  j ( N 1) 2
j
Pper (e )  Px (e ) * WR (e ) , WR (e )  e
2N sin( 2)
 2
1
PˆM (e j ) 
NU
 x ( n) w
n  
R ( n ) e  jn

N 1 2
1 1 
   W (e
j 2
U w( n )  ) d
N n 0 2N 

29
Example: x(n)  0.1sin(0.2n  1 )  1sin(0.3n  2 )  v(n)

N=128 N=128
Rectangular Window Hamming Window 30
Properties of the M-periodogram
 2
1
PˆM (e j )
NU
 x (
n  
n ) wR ( n ) e  jn

N 1 2
1
U
N
 w(n)
n 0

1
Bias: E{PˆM (e j )}  j j 2
Px (e )  W (e )
2NU

Resolution: window dependent

Variance: Var{PˆM (e j )}  Px2 (e j )

31
 Bartlett’s method (periodogram averaging)
L 1 2
1
ˆ j
Pper (e ) 
i

L
i
x (
n 0
n ) e  jn
; i  1 , 2 , ..., k

L Points L Points L Points

....
32
x1 (n) x2 (n) xk (n)
Properties of Bartlett’s method
k 1 L 1 2
1
ˆ j
PB (e ) 
N
 x ( n
i 0 n 0
 iL ) e  jn

1
Bias: E{PˆB (e j )}  Px (e j ) WB (e j )
2

2
Resolution: Res[ PˆB (e j )]  0.89k
N

ˆ j 1 2 j
Variance: Var{Pper (e )}  Px (e )
k
33
 Example:x(n)  1sin(0.25n  1 )  3sin(0.45n  2 )  v(n)

N  512 N  512 N  512

k 1 k 4 k  16 34
 Example:x(n)  1sin(0.25n  1 )  3sin(0.45n  2 )  v(n)

N  128 N  512 N  1024

k 4 k 4 k 4 35
Welch’s method (M-periodogram averaging)
xi (n)  x(n  iD) ; n  0 ,1,....., L  1 Overlap = L-D

....
36
 Properties of Welch’s method
k 1 L 1 2
1
PˆW (e j ) 
KLU
  x(n  iD )e
i 0 n 0
 jn

k 1
1 1 L 1
PˆW (e )   PˆM(i ) (e j )
j
U   w(n)
2
,
L i 0 L n 0

1
Bias E{PˆB (e j )}  j j 2
Px (e )  W (e )
2LU

Resolution Window dependent

Variance
ˆ j 9 L 2 j
Var{Pper (e )}  Px (e ) with 50% overlap 37
16 N
Example: x(n)  1sin(0.2  1 )  3sin(0.25  2 )  v(n)

N  512 N  512 , L  128

38
k 4 overlap  50% , hamming
 Resolution:x(n)  1sin(0.2n  1 )  3sin(0.25n  2 )  v(n)

N  512 , L  64 N  512 , L  128 N  512 , L  256

overlap  50% , hamming overlap  50% , hamming overlap  50% , hamming
39
windowing:x(n)  1sin(0.2n  1 )  3sin(0.25n  2 )  v(n)

N  512 , L  128 N  512 , L  128

40
overlap  50% , Rectangula r overlap  50% , Bartlett
 windowing:x(n)  1sin(0.2n  1 )  3sin(0.25n  2 )  v(n)

N  512 , L  128 N  512 , L  128 N  512 , L  128

overlap  50% , Hanning overlap  50% , Hamming overlap  50% , Blackman 41
Blackman-Tukey’s method (Periodogram smoothing)
•Note: Bartlett & Welch are design to reduce the variance if the priodogram
by averaging and modified it.

•Periodogram is computed by taking the Fourier transform of a consistent

estimate of the auto correlation sequence.

•For any finite data record of length N, the variance of rˆx (k ) will be large
for values of k that are close to N. for example:

1
rˆx ( N  1)  x( N  1) x(0) at lag k  n  1
N
•In Bartlett & Welch, the variance is decreased by reducing the variance of
autocorrelation estimate by averaging.
42
Blackman-Tukey’s method cont.

•In the Blackman-Tukey method, the variance is decreased by applying a

window to rˆx (k ) in order to decrease the contribution of the unreliable
estimates to the periodogram.
Specifically, the Blackman-Tukey spectrum estimation is:

M
PˆBT (e ) 
j
x
ˆ
r (
k  M
k ) w( k ) e  jk

•For example, if w(k) is a rectangular window extending from –M to M

with M<N-1 , then rˆx (k ) having the largest variance are set to zero and
consequently, the power spectrum estimation will have a smaller variance.

43
Properties of B-T’s method

M
PˆBT (e j )  x
ˆ
r (
k  M
k ) w( k ) e  jk

ˆ j 1
Bias E{PBT (e )}  Px (e j ) W (e j )
2

Resolution Window dependent

M
1
Variance Var{PˆBT (e )}  P (e )  w2 (k )
j
x
2 j

N M
44
windowing: x(n)  3sin(0.2  1 )  1sin(0.25  2 )  v(n)

N  512 , M  128 N  512 , M  128

Rectangula r Bartlett

45
N  512 , M  128 N  512 , M  128
Hanning Blackman
Performance comparisons

•We can summarized the performance of each technique in terms of

two criteria.

(I) Variability (which is a normalized variance)

Var{Pˆx (e j )}
 2
E {Pˆx (e j )}
(II) Figure of merit
  
•That is approximately the same for all of the nonparametric methods

46
 Summery
variability Resolution Figure of merit
  
2 2
Periodogram 1 0.89 0.89
N N
1 2 2
Bartlett 0.89k 0.89
k N N
91 2 2
Welch 1.28 0.72
8k L N
Blackman 2M 2 2
0.64 0.43
Tukey 3 N M N
47
***50% overlap and the Bartlett window***