Adaptive and Array Signal Processing

WS 2011 / 2012 Martin Haardt

Ilmenau University of Technology Communications Research Laboratory P.O. Box 10 05 65 D-98684 Ilmenau, Germany E-Mail: haardt@ieee.org

Ilmenau University of Technology Homepage: http://tu-ilmenau.de/crl Communications Research Laboratory

Adaptive and Array Signal Processing

Optional (3/1) subject for Elektrotechnik und Informationstechnik Ingenieurinformatik Medientechnologie Wirtschaftsingenieurwesen Mathematik Mandatory (3/2) subject for Master of Science in Communications and Signal Processing (MSCSP) Interested students from other majors are welcome to join.

Ilmenau University of Technology Communications Research Laboratory

Lectures and tutorials

Monday 09:00 h 10:30 h LdV-Hs 1 Tuesday 19:00 h 20:30 h LdV-Hs 2 mandatory for everybody 1 SWS of these will be used as tutorials (flexibly), the others as lecture

Monday (even weeks) 17:00 h 18:30 h LdV-Hs 1 additional tutorials, e.g., discuss previous homeworks mandatory for all MSCSP students (3/2) optional for all other students (3/1)
Monday (odd weeks) 17:00 h 18:30 h LdV-Hs 1 additional tutorials for the course Mobile Communications mandatory for all MSCSP students that have failed (or not taken) this exam in the summer term and need to repeat (take) it this winter
Ilmenau University of Technology Communications Research Laboratory

Hauptseminare Mobilkommunikation
Nach Absprache

Adaptive and Array Signal Processing

Nach Absprache

Ilmenau University of Technology Communications Research Laboratory

Motivation

This lecture provides an introduction to adaptive filters and array signal processing techniques. Several practical examples are taken from mobile communications. It is given in English to familiarize the students with the linguistic requirements of a global economy, where technical discussions between international partners are usually conducted in English.

Ilmenau University of Technology Communications Research Laboratory

Lectures

Fundamental concepts of adaptive filters and array signal processing are developed in class from temporal and spatial filters to high-resolution parameter estimation techniques important topics for several scientific disciplines A deep understanding of the underlying concepts is more important than learning mere figures and abbreviations by heart enables the students to develop their own research ideas and results Students are enabled to read and understand current research publications in the areas of adaptive filters and array signal processing use these concepts and results for your own research Please ask if something is not clear!
Ilmenau University of Technology Communications Research Laboratory

Goals (1)

Lecture in English to train your ability to understand, speak, write, and think in English necessity for your professional career combined with an optional student seminar (Hauptseminar) to deepen the material studied in class Homework assignments including Matlab exercises for a practical application of the material studied in class important part of this lecture (and, if desired, of the final grade) Style of the lecture development of the main concepts via handwritten notes less material, but more intensive coverage combined with a few selected PowerPoint slides as compared to a pure PowerPoint slide show (as it is the standard in many lectures)
Ilmenau University of Technology Communications Research Laboratory

Goals (2)
15 20 minutes review at the beginning of each lecture (endless overhead roll) with a special opportunity for questions from the students regarding the contents of the last lecture Continuous feedback questions for the students during the lectures to check whether the material has been understood Relationship to current topics in research and development student projects are a good opportunity to get some insight in this lecture: fundamentals required for the design of adaptive filters and array signal processing applications, e.g., mathematical background, adaptive spatial and temporal filters, (multi-dimensional) high-resolution parameter estimation

Ilmenau University of Technology Communications Research Laboratory

Goals (3)

Mixture of universal (timeless) principles that are applicable in several research areas and disciplines current hot topics from R&D in adaptive filters and array signal processing to enable you to read and understand current IEEE journal and conference publications in this area and to enable you to develop new research ideas and results that build on this published state of the art

Ilmenau University of Technology Communications Research Laboratory

Overview (1)
1 Introduction - Adaptive Filters - Single channel adaptive equalization (temporal filter) - Multi channel adaptive beamforming (spatial filter) 2 Mathematical Background 2.1 Wirtinger Calculus - Gradients - Differentiation with respect to a complex vector - Quadratic optimization with linear constraints (method of Lagrangian multipliers)

Ilmenau University of Technology Communications Research Laboratory

Overview (2)

2.2 Stochastic processes - Stationary processes - Time averages - Ergodic processes - Correlation matrices

Ilmenau University of Technology Communications Research Laboratory

Overview (3)
2.3 Linear algebra - Eigenvalue decomposition - Eigenfilter - Linear system of equations - Four fundamental subspaces - Singular value decomposition - Generalized inverse of a matrix - Projections - Low rank modeling

Ilmenau University of Technology Communications Research Laboratory

Overview (4)
3 Adaptive Filters 3.1 Linear Optimum Filtering (Wiener Filters) - Principle of Orthogonality - Wiener-Hopf equations - Error-performance surface - MMSE (minimum mean-squared error) - Canonical form of the error-performance surface - MMSE filtering in case of linear models

Ilmenau University of Technology Communications Research Laboratory

Overview (5)
3.2 Linearly Constrained Minimum Variance Filter - LCMV beamformer - Minimum Variance Distortionless Response (MVDR) spectrum: Capon's method - LCMV beamforming with multiple linear constraints 3.3 Generalized Sidelobe Canceler 3.4 Iterative Solution of the Normal Equations - Steepest descent algorithm - Stability of the algorithm - Optimization of the step-size 3.5 Least Mean Square (LMS) Algorithm
Ilmenau University of Technology Communications Research Laboratory

Overview (6)
4 High-Resolution Parameter Estimation - Data model (DOA estimation) - Eigendecomposition of the spatial correlation matrix at the receive array - Subspace estimates - Estimation of the model order 4.1 Spectral MUSIC - DOA estimation -Example: uniform linear array (ULA) - Root-MUSIC for ULAs - Periodogram - MVDR spatial spectrum estimation (review)
Ilmenau University of Technology Communications Research Laboratory

Overview (7)
4.2 Standard ESPRIT - Selection matrices - Shift invariance property 4.3 Signal Reconstruction - LS solution - MVDR / BLUE solution - Wiener solution (MMSE solution) - Antenna patterns 4.4 Spatial smoothing 4.5 Forward-backward averaging 4.6 Real-valued subspace estimation
Ilmenau University of Technology Communications Research Laboratory

Overview (8)
4.7 1-D Unitary ESPRIT - Reliability test - Applications in Audio Coding 4.8 Multidimensional Extensions - 2-D MUSIC - 2-D Unitary ESPRIT - R-D Unitary ESPRIT 4.9 Multidimensional Real-Time Channel Sounding 4.10 Direction of Arrival Estimation with Hexagonal ESPAR Arrays

Ilmenau University of Technology Communications Research Laboratory

Overview (9)
5 Tensor-Based Signal Processing 5.1 Fundamental Concepts of Tensor Algebra 5.2 Tensor Decomposition 5.3 Tensors in Signal Processing Applications 6 Maximum Likelihood Estimators 6.1 Maximum Likelihood Principle 6.2 The Fisher Information Matrix and the Cramr-Rao Lower Bound - Efficiency - CRLB for 1-D direction finding applications - Asymptotic CRLB 7 Conclusions
Ilmenau University of Technology Communications Research Laboratory

References

G. Strang, Introduction to Linear Algebra, Wellesley-Cambridge Press, Wellesley, MA, 1993. G. Strang, Linear Algebra and its applications, Harcourt Brace Jovanovich, San Diego, CA, 3rd edition, 1988. S. Haykin, Adaptive Filter Theory, Prentice-Hall, 4th edition, 2002. H. L. Van Trees, Optimum Array Processing, John Wiley & Sons, Inc., New York, NY, 2002. L. L. Scharf, Statistical Signal Processing, Addison-Wesley Publishing Co., 1991. T. K. Moon and W. C. Stirling, Mathematical Methods and Algorithms for Signal Processing, Prentice-Hall, 2000. A. H. Sayed, Fundamentals of Adaptive Filtering, John Wiley & Sons, Inc., New York, NY, 2003. S. M. Kay, Fundamentals of Statistical Signal Processing, Estimation Theory, Prentice-Hall, Englewood Cliffs, N.J., 1993. M. Haardt, Efficient One-, Two-, and Multidimensional High-Resolution Array Signal Processing, Shaker Verlag, 1997. S. Haykin and M. Moher, Modern Wireless Communications, Pearson Education, Inc., 2005. T. Kaiser, A. Bourdoux, H. Boche, editors, Smart Antennas State of The Art, Hindawi Publishing Corporation, 2005. A. Paulraj, R. Nabar, and D. Gore, Introduction to Space-Time Wireless Communications, Cambridge University Press, 2003. Ilmenau University of Technology Communications Research Laboratory

Homeworks (1)

Homework assignments will be given to you as the semester progresses. Some typical homework assignments involve: showing some results discussed in class

theoretical proofs
simulation studies using Matlab examining the performance of a system extracting key parameters from measurements

Most of the topics covered by the course are best learned by working with the problems through a combination of computer simulations and theoretical analysis. The homework is therefore one of the most important parts of the course.
Ilmenau University of Technology Communications Research Laboratory

Homeworks (2)

Cooperative group study on the homework is encouraged, but simply copying someone else's work is unethical and will leave you unprepared for the final exam. Significant insight can be gained by studying with another student or in a group, provided you discipline yourself to find your own solutions first before comparing results. Rely on other's help only when you have exhausted all of your own ideas or have made no progress for an extended period of time.

Ilmenau University of Technology Communications Research Laboratory

Projects (1)
The projects...

encourage professional preparation

provide research and development opportunities

provide opportunities for students to work in teams

The projects give the students the ability to...

apply knowledge of mathematics, science, and engineering function on multi-disciplinary levels formulate and solve engineering problems communicate effectively and use engineering English use the techniques, skills, and modern engineering tools like Matlab which are necessary for engineering practice
Ilmenau University of Technology Communications Research Laboratory

Projects (2)
Each student ...

prepares one lesson on the material covered in his/her project develops, in detail, some portion of the fundamental theory in his/her selected topic area carefully selects what subset of the material is fundamental to the understanding of the remainder of the presentation as not all aspects of the papers can be explained in detail

Ilmenau University of Technology Communications Research Laboratory

Projects (3)
The students will...

gain knowledge of contemporary issues get an insight in the proposed topics. Moreover, they should present their topics and research results to the class. This will be done at the end of the semester.

Grading of the projects

Abstract Presentation structure and understanding of the material Contents and level of the presented research Answers to questions Simulation results Slides and didactical preparation
Ilmenau University of Technology Communications Research Laboratory

Student Projects / Hauptseminar

See separate slide set

Ilmenau University of Technology Communications Research Laboratory

Deadlines for the Student Projects

Several milestones are set to assist students in completing a good presentation on time:

October 24, 2011 November 21, 2011

Choice of the topics Abstract, defining the problem area (2 4 pages) Hand in drafts of the slides

feedback will be provided

December 12, 2011

feedback will be provided

from January 09, 2012 Presentations

Ilmenau University of Technology Communications Research Laboratory

Grading Options
MSCSP options in blue
No homeworks, no project No Homework and student project No homeworks but a Hauptseminar Homework, no Hauptseminar Homeworks and Hauptseminar Homeworks and student project

Homework

Student project

Hauptseminar

Final exam 100%

30 %
Schein 30% 30% 30% 30% Schein

70%
100% 70% 70% 40%

Ilmenau University of Technology Communications Research Laboratory

1. Introduction

Ilmenau University of Technology Communications Research Laboratory

System identification (channel estimation)

training sequence (n) (n)

(n)

(channel)
Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

Inverse modeling (adaptive equalization)

optional training sequence (n) (n) (channel) (n) (n)

Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

Prediction

adaptive filter output (n) (n) (n) (n) prediction error

Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

Interference cancelation

(n) (n) (n) (n)

Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

Adaptive noise cancelation

Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

Single channel adaptive equalization

Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

Parameter Estimation in Array Signal Processing (1)

Estimate the directions of arrival (DOAs) and the location of electromagnetic sources (radar) or acoustic sources (sonar) active radar antenna array is used for the transmission and reception of electromagnetic signals active sonar systems for the detection and localization of submarines as well as the estimation of other parameters (e.g., speed) Radio astronomy passive system to detect celestial objects and to estimate their characteristics

Detection of seismic waves in geophysics estimate the propagation velocity of these seismic waves and characterize the geological nature of the terrain
Ilmenau University of Technology Communications Research Laboratory

Parameter Estimation in Array Signal Processing (2)

RF communications parameter estimation based on channel sounder measurements development of realistic channel models (more details later) Bio-medical imaging use circular antenna arrays to focus the energy on transmit and receive multi-sensor ECG recordings reveal information about the state of the heart multi-sensor EEG recordings may be used to localize activities in the brain Localization of aero-acoustic sources e.g., on the surface of a car in a wind tunnel
Ilmenau University of Technology Communications Research Laboratory

Examples of Passive Sonar Systems (1)

Submarine mounted hydrophone array

Source: H. L. Van Trees, Optimum Array Processing. John Wiley & Sons, Inc., New York, NY, 2002.

Ilmenau University of Technology Communications Research Laboratory

Examples of Passive Sonar Systems (2)

Towed hydrophone array

Source: H. L. Van Trees, Optimum Array Processing. John Wiley & Sons, Inc., New York, NY, 2002.

Ilmenau University of Technology Communications Research Laboratory

Seismic Experiment

Source: H. L. Van Trees, Optimum Array Processing. John Wiley & Sons, Inc., New York, NY, 2002.

Ilmenau University of Technology Communications Research Laboratory

Directional Transmission and Reception at the Base Station via Smart Antennas

antenna array at the base station (& optionally at the terminal) + baseband signal processing increased antenna gain & diversity gain towards the desired user reduced interference in the other directions spatial multiplexing gain significant improvements in capacity, coverage, and quality of service Ilmenau University of Technology Communications Research Laboratory

Multi channel adaptive beamforming (1)

u1(n) u2(n) u3(n) u4(n) u5(n)

M=5
Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

Multi channel adaptive beamforming (2)

u1(n)

q
u2(n) u3(n) u4(n) u5(n)

M=5
Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

2. Mathematical Background

Ilmenau University of Technology Communications Research Laboratory

2.1 Wirtinger Calculus

Gradients

Ilmenau University of Technology Communications Research Laboratory

Wirtinger Calculus

Differentiation with respect to a complex vector

alternatively define main idea behind Wirtinger calculus introduce conjugate coordinates

Ilmenau University of Technology Communications Research Laboratory

Example

Ilmenau University of Technology Communications Research Laboratory

Wirtinger Calculus

(ensemble of differentiable functions) Definition of a gradient: change of the value of the function for an infinitesimal change of the parameter values (direction of the greatest ascent) corresponds to the maximum rate of change

Ilmenau University of Technology Communications Research Laboratory

Example 1

Ilmenau University of Technology Communications Research Laboratory

Example 1 (continued)

Ilmenau University of Technology Communications Research Laboratory

Example 2

Ilmenau University of Technology Communications Research Laboratory

Example 2 (continued)

Ilmenau University of Technology Communications Research Laboratory

Selected References
(1) K. Kreutz-Delgado, "The Complex Gradient Operator and the CRCalculus", arXiv:0906.4835. UCSD-ECE275CG-S2009V1.0, available at http://arxiv.org/abs/0906.4835v1 (2) John W. Brewer, "Kronecker Products and Matrix calculus", IEEE Trans. on Circuits and Systems, vol. CAS-25 (9), Sep 1978. (3) K. B. Petersen and M. S. Pedersen, "The Matrix Cookbook", Technical University of Denmark, Version 20081110, Oct 2008, available at http://www2.imm.dtu.dk/pubdb/p.php?3274.

Ilmenau University of Technology Communications Research Laboratory

Single channel adaptive equalization

Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

The four fundamental subspaces of the matrix A

Source: G. Strang, Introduction to Linear Algebra, 1993.

Ilmenau University of Technology Communications Research Laboratory

Multi channel adaptive beamforming (1)

u1(n) u2(n) u3(n) u4(n) u5(n)

M=5
Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

Multi channel adaptive beamforming (2)

u1(n)

q
u2(n) u3(n) u4(n) u5(n)

M=5
Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

SVD of Matrices

Full SVD

Economy size SVD

Ilmenau University of Technology Communications Research Laboratory

The four fundamental subspaces of the matrix A

Source: G. Strang, Introduction to Linear Algebra, 1993.

Ilmenau University of Technology Communications Research Laboratory

A x = b in the four fundamental subspaces

Source: G. Strang, Introduction to Linear Algebra, 1993.

Ilmenau University of Technology Communications Research Laboratory

A+ inverts A from its row space to its column space

Projection into column space

Source: G. Strang, Introduction to Linear Algebra, 1993.

Ilmenau University of Technology Communications Research Laboratory

A x = b in the four fundamental subspaces

Projection into row space

Source: G. Strang, Introduction to Linear Algebra, 1993.

Ilmenau University of Technology Communications Research Laboratory

Efficient Solution of the Least Squares Problem

Ilmenau University of Technology Communications Research Laboratory

SVD of Matrices

Full SVD

Economy size SVD

Low-rank approximation

Ilmenau University of Technology Communications Research Laboratory

3. Adaptive Filters

Ilmenau University of Technology Communications Research Laboratory

Single channel adaptive equalization

Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

Channel Estimation

Source: A. H. Sayed, Fundamentals of Adaptive Filtering, Wiley, 2003

Ilmenau University of Technology Communications Research Laboratory

Block Data Estimation

Source: A. H. Sayed, Fundamentals of Adaptive Filtering, Wiley, 2003

Ilmenau University of Technology Communications Research Laboratory

Multi channel adaptive beamforming (1)

u1(n) u2(n) u3(n) u4(n) u5(n)

M=5
Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

Multi channel adaptive beamforming (2)

u1(n)

q
u2(n) u3(n) u4(n) u5(n)

M=5
Source: S. Haykin, Adaptive Filter Theory, Prentice Hall, 3rd ed., 1996.

Ilmenau University of Technology Communications Research Laboratory

Uniform Linear Array (ULA) of omnidirectional sensors

Ilmenau University of Technology Communications Research Laboratory

Adaptive FIR filter

Source: S. Haykin, Adaptive Filter Theory, 1996.

Ilmenau University of Technology Communications Research Laboratory

Bank of cross-correlators

Source: S. Haykin, Adaptive Filter Theory, 1996.

Ilmenau University of Technology Communications Research Laboratory

Steepest Descent Algorithm: Iterative minimization of a quadratic form (1)

Ilmenau University of Technology Communications Research Laboratory

Steepest Descent Algorithm: Iterative minimization of a quadratic form (2)

350 300 250 200 150 100 50 0 -10

-5

10

-10

-5

10

Ilmenau University of Technology Communications Research Laboratory

Steepest Descent Algorithm: Iterative minimization of a quadratic form (3)

400 200 0 -10 -5 0 5 10 -10 -5 0 5

10

Ilmenau University of Technology Communications Research Laboratory

Steepest Descent Algorithm (1)

Ilmenau University of Technology Communications Research Laboratory

Steepest Descent Algorithm (2)

Ilmenau University of Technology Communications Research Laboratory

Steepest Descent Algorithm (3)

Ilmenau University of Technology Communications Research Laboratory

Adaptive weight-control mechanism

Source: S. Haykin, Adaptive Filter Theory, 1996.

Ilmenau University of Technology Communications Research Laboratory

Steepest Descent vs. LMS Algorithm

Cost function Cost function (contour plot)

Ilmenau University of Technology Communications Research Laboratory

Learning Curves
Single realization

Ensemble average (10,000 realizations)

Ilmenau University of Technology Communications Research Laboratory

Adaptive Implementation of the Generalized Sidelobe Canceller

H wq

H Ca

H w a (n)

Source: S. Haykin, Adaptive Filter Theory, 2002.

Ilmenau University of Technology Communications Research Laboratory

Adaptive Channel Estimation

Source: A. H. Sayed, Fundamentals of Adaptive Filtering, Wiley, 2003

Ilmenau University of Technology Communications Research Laboratory

Linear Equalization of an FIR Channel

Source: A. H. Sayed, Fundamentals of Adaptive Filtering, Wiley, 2003

Ilmenau University of Technology Communications Research Laboratory

Adaptive Equalization
Source: A. H. Sayed, Fundamentals of Adaptive Filtering, Wiley, 2003

Ilmenau University of Technology Communications Research Laboratory

Adaptive Linear Equalizer with 2 Modes

Source: A. H. Sayed, Fundamentals of Adaptive Filtering, Wiley, 2003

Ilmenau University of Technology Communications Research Laboratory

Decision-Feedback Equalizer

Source: A. H. Sayed, Fundamentals of Adaptive Filtering, Wiley, 2003

Ilmenau University of Technology Communications Research Laboratory

Adaptive Decision-Feedback Equalization

Source: A. H. Sayed, Fundamentals of Adaptive Filtering, Wiley, 2003

Ilmenau University of Technology Communications Research Laboratory