Introduction to Digital Communication Systems

Lecture Notes for ECE 361

Fall 1995

Dilip V. Sarwate
Department of Electrical and Computer Engineering
University of Illinois at Urbana-Champaign
Urbana, Illinois 61801

© 1995 by Dilip V. Sarwate

All rights reserved. No part of this manuscript may be reproduced, stored in a retrieval system,
or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or
otherwise without the prior written permission of the author.
 Introduction to Digital Communication Systems: Lecture Notes for ECE 361, Fall 1995

Note to ECE 361 students

These optional lecture notes sold through the IEEE Student Branch are intended as a supplement
to the required textbook for ECE 361: Introduction to Digital Communications by M. B. Pursley,
(also sold through the IEEE Student Branch.) The lecture notes include the text and diagrams
that will appear on the viewgraphs that I will use in teaching ECE 361 in Fall 1995. The lecture
notes also include much additional material in the form of comments and remarks on the material
presented on the viewgraphs, as well as several exercises. This additional material is intended to
help you understand the concepts presented on the viewgraphs. In some instances, the comments
will remind you of a key fact that will make it clear how Step 6 on the viewgraph follows from
Step 5. The comments may have a few details that have been skipped on the viewgraph itself, or
(horrors!) they may suggest that you work out the details of some derivation all by yourself.
Many of these comments (usually in paraphrased form) will also be made orally during the
lectures. Thus, the inclusion of the comments in the lecture notes saves you the trouble of
scribbling down these pearls of wisdom. Of course, some of these comments will not be made
orally, usually because of sheer forgetfulness on my part but occasionally because of a lack of
time. Thus, the inclusion of the comments in these lecture notes ensures that what I forgot to say
or did not have time to say is nonetheless available to you. Please keep in mind that, regardless
of the reason for the omission, the remarks included in these lecture notes but not said out loud in
class are just as important as the ones stated in a ringing voice, and you are urged not to omit
such unstated facts when studying for the course.

In these lecture notes, the material that will appear on the viewgraphs is printed with borders
around it. The additional remarks and comments are distinguished by not having any borders.
The lecture notes are also printed with wide outside margins to provide you with space for
making your own notes or for doodling when the lectures are more boring than usual.

Dilip V. Sarwate
Urbana, Illinois
August 1995

iii
 Introduction to Digital Communication Systems: Lecture Notes for ECE 361, Fall 1995

Contents
Introduction to Digital Communication Systems............................................................................................. 1
Analog vs Digital Communication Systems........................................................................................... 1
Modern Digital Communicaton Theory.................................................................................................. 3
Block Diagram of a Digital Communication System............................................................................. 4
Probability Review .............................................................................................................................................. 10
Notation and Axioms.............................................................................................................................. 10
Conditional Probability ............................................................................................................ 12
Bayes’ Formula ........................................................................................................................ 14
Stochastic Independence .......................................................................................................... 15
Random Variables .................................................................................................................................. 17
Expectation............................................................................................................................... 18
Gaussian Random Variables .................................................................................................... 20
Many Random Variables ........................................................................................................................ 21
Conditional Distributions ......................................................................................................... 23
Jointly Gaussian Random Variables ........................................................................................ 28
Limit Theorems ...................................................................................................................................... 29
Markov’s and Chebyshev’s Inequalities; the Chernoff Bound ................................................ 29
The Laws of Large Numbers.................................................................................................... 31
The Central Limit Theorem .................................................................................................................... 32
Random Processes............................................................................................................................................... 33
Definition and Basic Properties .............................................................................................................. 33
Distribution and Density Functions ........................................................................................................ 35
Mean, Autocorrelation and Covariance.................................................................................................. 36
Stationary and Wide-Sense-Stationary Random Processes.................................................................... 38
Cross-Correlation and Cross-Covariance Functions .............................................................................. 41
White Noise Processes............................................................................................................................ 42
Random Processes in Linear Systems ............................................................................................................... 43
Mean-Square Convergence vs Stochastic Mean-Square Convergence .................................................. 44
Interchange of Expectation and Integration Operations ......................................................................... 46
The Mean of the Output Process ............................................................................................................ 47
Cross-correlation between Input and Output Processes.......................................................................... 47
Autocorrelation Function of the Output Process .................................................................................... 48
Spectral Analysis of Random Processes................................................................................................. 51
Power Spectrum of a Random Digital Signal......................................................................................... 52
Baseband Digital Communication Systems ...................................................................................................... 55
Binary Communication System.............................................................................................................. 55
Signal Output from Filter ......................................................................................................... 57
Noise Output from Filter .......................................................................................................... 58
Thresholds and Error Probabilities........................................................................................... 58
Conditional Error Probabilities ................................................................................................ 59
Two Criteria for Choosing the Threshold............................................................................................... 60
Minimax Threshold Criterion................................................................................................................. 60
Summary for Minimax Threshold Criterion ............................................................................ 62
Minimum Average Error Probability (Bayes’) Criterion........................................................................ 66
Properties of Bayes’ Threshold ................................................................................................ 69
Properties of Bayes’ Error Probability..................................................................................... 69
Summary for Minimax and Bayes’ Criteria............................................................................................ 72
Matched Filters....................................................................................................................................... 74
The Schwarz Inequality............................................................................................................ 74
Proof......................................................................................................................................... 75
Application to Matched Filter Design ...................................................................................... 76
Error Probabilities for Orthogonal and Antipodal Signaling ................................................... 80
Frequency-Domain Interpretation of the Matched Filter ......................................................... 83
Transmission of a Sequence of Bits ......................................................................................... 86

v
 Introduction to Digital Communication Systems: Lecture Notes for ECE 361, Fall 1995

Correlation Receivers.............................................................................................................................. 89
Comparison of Matched Filter and Correlation Receiver Outputs......................................................... 91
Summary of Matched Filters and Correlation Receivers........................................................................ 94
Signal Design.......................................................................................................................................... 95
Pre-whitening Filters .............................................................................................................................. 101
Signal Space Concepts ........................................................................................................................................ 104
Orthogonal and Orthonormal Functions................................................................................................. 104
Minimum Mean-Square Error Representations........................................................................ 105
Complete Orthonormal Sets..................................................................................................... 106
Gram-Schmidt Orthonormalization Procedure ........................................................................ 107
Orthonormal Representations................................................................................................................. 108
Signal Representation............................................................................................................... 109
Noise Representation................................................................................................................ 112
Vector Channels...................................................................................................................................... 113
Decision Regions .................................................................................................................................... 114
Binary Signaling..................................................................................................................................... 115
Maximum-Likelihood Decision Regions................................................................................. 116
Bayesian Decision Regions ...................................................................................................... 120
Summary for Binary Signaling ................................................................................................ 123
M-ary Signaling ...................................................................................................................................... 125
Preference Regions and Decision Regions............................................................................... 125
Symbol Error Probabilities....................................................................................................... 130
Two Receiver Implementations................................................................................................ 132
Union Bound on Error Probabilities......................................................................................... 134
Nearest-Neighbor Approximation for Error Probabilities........................................................ 136
Bit Error Probabilities .............................................................................................................. 138
Relationship between Bit Error Probability and Symbol Error Probability............................. 140
One-dimensional M-ary Signaling.......................................................................................................... 143
Symbol and Bit Error Probabilities .......................................................................................... 144
Signal Energy ........................................................................................................................... 145
Asymptotic Behavior for Large M........................................................................................... 146
Two-dimensional M-ary Signaling......................................................................................................... 147
Symbol and Bit Error Probabilities .......................................................................................... 149
Signal Energy ........................................................................................................................... 151
Asymptotic Behavior for Large M........................................................................................... 152
M-ary Phase-Shift Keying ...................................................................................................................... 153
Symbol Error Probabilities....................................................................................................... 154
Summary for One- and Two-Dimensional M-ary Signaling.................................................................. 155
Multidimensional M-ary Signaling......................................................................................................... 156
Symbol and Bit Error Probabilities .......................................................................................... 156
Signal Energy ........................................................................................................................... 157
Asymptotic Behavior for Large M........................................................................................... 158
Bandwidth of a Digital Signal ................................................................................................................ 159
The Landau-Pollak Theorem.................................................................................................... 160
M-ary Orthogonal Signaling................................................................................................................... 161
Probability of Correct Decision and Probability of Error ........................................................ 162
Union Bound on the Probability of Error................................................................................. 163
Probability of Bit Error ............................................................................................................ 164
Asymptotic Behavior for Large M........................................................................................... 166
Simplex or Trans-Orthogonal M-ary Signaling...................................................................................... 170
Bi-Orthogonal M-ary Signaling.............................................................................................................. 170
Union Bound of the Probability of Error.................................................................................. 173
Bit Error Probability................................................................................................................. 174
Generation of Orthogonal, Transorthogonal, and Bi-orthogonal Signals............................................... 177
FSK, PPM, and Hadamard Signal Sets .................................................................................... 177
Fast Walsh-Hadamard Transforms........................................................................................... 178
Summary for Multidimensional M-ary Signaling .................................................................................. 180
Optimality of Linear Receivers on Additive Gaussian Noise Channels................................................. 183

vi