A New Method for Calculating Symbol Error Probabilities of Two-Dimensional Signalings in Rayleigh Fading with Channel Estimation Errors 1

Xiaodai Dong, Member, IEEE, Norman C. Beaulieu, Fellow, IEEE Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Alberta, Canada T6G 2V4 Email: xdong, beaulieu @ee.ualberta.ca Tel: 780 492-6989 Fax: 780 492-0194

1 This

work was presented in part at the 2002 IEEE Symposium on Advances in Wireless Communications, Victoria, Canada, September

2002, and the 2003 IEEE International Conference on Communications (ICC2003), Anchorage, USA, May 2003.

Abstract A general analytical framework for evaluating the performance of coherent two-dimensional signaling in frequency at Rayleigh fading with channel estimation is proposed in this paper. A new and simple analytical expression for the symbol error rate of an arbitrary polygonal two-dimensional constellation in Rayleigh fading in the presence of channel estimation errors is presented. This framework is applicable to many current channel estimation methods such as pilot symbol assisted modulation and minimum mean square error estimation where the fading estimate is a complex Gaussian random variable correlated with the channel fading. The sensitivity of various high-level two-dimensional signaling formats to static and dynamic channel amplitude and phase estimation errors in Rayleigh fading can be easily studied using the derived formula. The new exact expression makes it possible to optimize constellation parameters and various parameters associated with channel estimation schemes. It also provides insights into choosing an appropriate signaling format for a fading environment with practical channel estimation methods used at the receiver. Index Terms Channel estimation errors, fading channels, high order modulation, imperfect channel estimation, minimum mean square error (MMSE), M -ary phase shift keying (MPSK), pilot symbol assisted modulation (PSAM), quadrature amplitude modulation (QAM), symbol error probability, two-dimensional (2-D) modulation formats.

I. INTRODUCTION The perfect coherent detection of arbitrary polygonal two-dimensional (2-D) signaling in frequency at fading channels has been well studied. Tractable analytical results for determining the performances of 2-D signaling schemes are available for various fading models [1]-[2]. These results assume perfect fading channel state information. In reality, channel state information is only available through estimation algorithms with inherent channel estimation errors. Therefore, it is of practical interest and importance to study the performances of coherent signaling formats in the presence of channel estimation errors. Common channel estimation methods studied in the literature are pilot symbol assisted modulation (PSAM) and minimum mean square error (MMSE) estimation. A classical paper on pilot symbol assisted modulation for Rayleigh fading was presented by Cavers in 1991 [3] where he derived the optimum Wiener

interpolation lter to minimize the variance of the estimation error and studied the system performance for binary phase shift keying (BPSK), quaternary phase shift keying (QPSK) and 16-ary quadrature amplitude modulation (QAM). The optimum Wiener lter requires a priori information of the channel and is computationally complex. Therefore, several suboptimal interpolation lters have been proposed among which the sinc interpolator is most widely used due to its simple implementation and close-to-optimum performance [4]. The bit error rate (BER) of pilot symbol assisted M -ary QAM (MQAM) in Rayleigh fading using the sinc interpolator has been analyzed in [5]. An upper bound to the symbol error rate (SER) for pilot symbol assisted QAM in Rayleigh fading was presented in [6], and the optimum MMSE interpolator using an innite number of pilot symbols was examined. The performance of MMSE channel estimation has been investigated for M -ary PSK (MPSK) and MQAM in Rayleigh fading in [7] and for Ricean fading in [8]. The analyses in these two papers employed the orthogonality between the channel estimate and the channel estimation error. Their work has been further extended by Wilson and Ciof [9] to eliminate the orthogonality requirement. The probability density functions (PDFs) of the receiver decision variables for Rayleigh fading channels with maximal ratio diversity combining, and for single branch Ricean fading channels in the presence of imperfect channel estimation were derived. The PDFs were employed to calculate the SER and BER of 16-QAM with channel estimation errors. However, the analysis in [9] is only applicable to MQAM. In summary, results are only available for MPSK and MQAM with small M values; results for arbitrary 2-D signaling are not available. Furthermore, there has been no analysis that is able to unify these previously published results and to account for more general case of arbitrary 2-D signalings. We present in this paper a general method of analysis for determining symbol error probabilities of arbitrary 2-D signalings in Rayleigh fading with static or dynamic channel estimation errors. It is applicable to any fading estimate which is jointly Gaussian with the actual fading. This includes, but is not limited to, MMSE and pilot symbol assisted modulation schemes. Our analytical method is new and applies very generally to modulation formats not previously studied as well as to modulation formats previously studied using other methods which are not applicable to the general modulations considered here. The paper is organized as follows. Section II derives the error probability expression for coherent 2-D signal-

ing in Rayleigh fading with channel estimation errors. Section III applies the newly derived formula to the analysis of dynamic estimation errors when PSAM and MMSE estimation are employed as well as to the analysis of static estimation errors. Section IV discusses the numerical SER results for various 8-ary and 16-ary constellations that are of practical interest or are commonly studied in the literature. Finally, conclusions are given in Section V. II. ERROR PROBABILITY OF 2-D SIGNALING This section begins with the system modeling. We then present our symbol error probability analysis of 2-D signalings in Rayleigh fading with channel estimation errors. Assume perfect symbol timing, intersymbol interference (ISI) free transmission, and slow channel fading that is almost constant over one symbol duration. The received signal z at any symbol interval given signal s i

sIi

jsQi i

1 M (1)

transmitted can be expressed as z gsi n where the zero-mean Gaussian random variable (R.V.) g

gI

jgQ represents the complex fading in

troduced by the channel with variance Es in both of its real and imaginary parts, and n

nI

jnQ is a

zero-mean Gaussian noise R.V. with variance N0 in both of its real and imaginary parts. The fading g is independent of the additive Gaussian noise n. Signal si is assumed to be normalized to yield unit average energy of the constellation. Various channel estimation methods, for instance, pilot symbol assisted modulation and minimum mean square error estimation, generate a fading estimate g that is also Gaussian and correlated with the true fading g. The specic expression for g depends on the particular estimation method used. The most often used method for calculating the probability of error of a signaling format is based on the conditional error probability given the channel fading g and the fading estimate g . Under this conditioning, the problem is similar to the well known additive white Gaussian noise (AWGN) case. The probability of error is then obtained by averaging the conditional error probability over the joint distribution of g and g . Previous publications [5]-[6] follow this approach. However, in the general case of arbitrary 2-D constellations, this approach often leads to an intractable analysis. Averaging over the joint

e j distribution of g e j and g usually involves a four-fold nested integration, and the conditional

 for arbitrary 2-D signaling usually has another level of integration. That is, probability P e  P e
2 2



 f  d d d d P e    

(2)

 requires a single integration for general 2-D signaling as seen in [2], [1]. and P e  We propose in this paper a completely different approach to solving the problem and arrive at a surprisingly simple yet general expression for the symbol error probability of arbitrary 2-D signaling in Rayleigh fading with channel estimation errors. We begin with dening a decision variable as D And then, t

z g

g n si g g

si t

(3)

z g


si D


si

(4)

where t

re j can be considered as an equivalent noise term that is superimposed on the signal s i and

includes the effects of both the fading estimation error and the AWGN. Assuming equally-likely transmission of signals, the maximum likelihood (ML) decision rule of a constellation in AWGN and in fading with perfect channel sate information is equivalent to the minimum Euclidean distance criterion. In the case of imperfect channel estimation, however, minimum Euclidean distance rule is no longer the optimum. Nevertheless, in practice a receiver will likely use the estimated fading as if it were perfect channel state information and draw up decision boundaries as in the AWGN case. This is because the receiver may not have the necessary information, such as the maximum Doppler frequency, the signal-to-noise ratio (SNR) and so on, needed in the PDF of the decision variable to determine the optimum decision boundaries. Therefore, we consider the practical case where the minimum Euclidean distance rule is used in the receiver. If D falls into the decision region of signal si , a correct decision will be made; otherwise, a symbol error will occur. The decision variable D was also written as the sum of the signal and an equivalent noise term in [7] and [8]. Furthermore, the probability density function of the noise term t was obtained from the conditional

PDF as ft r 

0

 f  d d d d ft r    

(5)

which can be simplied for MMSE estimation as discussed in [7] and [8]. However, the four-fold nested integration in (5) usually cannot be solved for other practical channel estimation schemes. Here we present a new derivation of the PDF of t . Now, recognizing the fact that the decision variable D in (3) is the ratio of two complex-valued Gaussian random variables z and g , we rst derive the distribution of the ratio of two complex Gaussian R.V.s X

z and Y

g . Dene variable D in a general form as (6)

X Y

where X and Y are correlated zero-mean complex-valued Gaussian random variables. The joint PDF of zero-mean complex Gaussian R.V.s X and Y was given by Wooding in a simple and compact form as [10] fX  Y X Y 

 L 

exp


VH L  1 V 

(7)

where V  X Y "! H with

#

representing complex conjugation and AH the Hermitian transpose of matrix

'(

A. Matrix L is the Hermitian covariance matrix given by L %$& mxx mxy mxy myy (8)

where mxx E  X X )! , myy E  YY "! and mxy E  XY ! are the second moments of the zero-mean complexvalued Gaussian R.V.s X and Y . Eqn. (7) can also be written as fX  Y X Y 

 L 

exp 


 L 

myy  X  2 mxx  Y  2


mxy XY


mxy X Y )!

(9)

and in polar coordinates X

rx e jx and Y rx ry  L  2
1

r y e j y ,
  L 
1

f X  Y rx x ry y 

exp 

2 2 myy rx mxx ry

2 0 mxy rx ry e j 1 x 

y 23

4!

(10)

To obtain the PDF of D rd e jd

X Y,

we dene another complex variable F

r f e j f

Y . Then, we have

the following relationships rd

d rf f

rx ry x


(11a) y (11b) (11c) (11d)

ry y

The joint PDF of D and F is obtained from (10) as f D  F rd d r f f 

f X  Y rd r f d f r f f  J j d 3  r 2 2 m myy rd xx 2 0 mxy rd e rd 3 f  r f exp 5 2  L  L 6 

1 ry 1 rf

(12) . The PDF of

where J is the Jacobian of the transformation (11) and it can be shown that  J  variable F , f D rd d 

2

the complex variable D is then obtained by integrating the joint density (12) with respect to the complex

 L

fD  F rd d r f f  d f dr f mxx


2 myy rd

rd 2 0 mxy rd e jd 3

 2

(13)

The next step is to derive the PDF of the combined noise term t D Dx jDy given by (13) in rectangular coordinates, we have f D Dx Dy 
 L

re j in (4). Rewriting the PDF of

2  myy Dx

D2 y

mxx


1 2 0 mxy 3 Dx


2 0 mxy 3 Dy ! 2

(14)

It can be shown that (14) is equivalent to [9, eqn. (3)] but (14) is derived using a different method. Moreover, our later application of Craigs method to general 2-D signaling schemes will require the PDF of the

combined noise term t in polar coordinate form. Since t

tx sIi

jty D


si , one has (15a) (15b)

tx

Dx
 

ty The Jacobian of transformation (15) is  J 

Dy

sQi

1, and thus the PDF of t can be written as 2 myy sIi



ft tx ty 

 L

myy  si  2 mxx

2  myy tx

2 ty 

0 mxy 3  tx 2 myy sQi



2 0 mxy 3 sIi

2 0 mxy 3 sQi !


0 mxy 3  ty (16)

Furthermore, the PDF of the complex-valued variable t expressed in polar coordinates can be obtained
2 t 2 and from (16) by another change of variables r 87 tx y tan  1

ty 9 tx  as (17a)

ft r 

 L

cr2

r b  r a

where a

myy  si  2 mxx


2 0 mxy 3 sIi
 

2 0 mxy 3 sQi 0 mxy 3 cos



(17b) 0 mxy 3 sin  (17c) (17d)

b  c

2 myy sIi cos myy sQi sin myy

Once the PDF (17) of the equivalent noise term t accounting for both the channel estimation error and the additive white Gaussian noise is available, we apply Craigs method [11], [1], [2] to analyze the probability of symbol error for arbitrary 2-D signaling. The main idea of Craigs method is to work directly with the PDF of the noise term that is superimposed onto the signal si and divide erroneous decision regions into smaller subregions. Each subregion can be conveniently represented by polar coordinates [11]. Since we have expressed the decision variable D as the sum of si and t , we shift the origin of the coordinates to signal si

sIi

jsQi to form new coordinates and work directly with t . Fig. 1 illustrates how to partition

the erroneous decision region of signal si into subregions. Following analysis similar to that presented in [2], the probability of error for the j-th erroneous subregion is given by Pe  j
@ @ @
2 A j

R 

x j sin j sin B 1  j  j 
@ 

@ 1 A j

R 1 2

ft r  drd @
@ @

(18a) (18b)

where angles 1  j 2  j j and distance x j are parameters specic to the geometry of the j-th erroneous subregion and are illustrated in Fig. 1 for different erroneous subregions. It can be seen from (17) that the PDF of the combined noise t is no longer Gaussian and does not have circular symmetry as in the perfect channel state information case. The integration limits 1  j and 2  j must be angles related to the j-th subregion in the new coordinates (si is the origin). In the open decision region case shown in Fig. 1(b), 1  j is dened as the angle formed by the horizontal axis of the new coordinates and the line from signal s i to the intersection of two decision boundaries. The absolute value is needed in (18a) as 1  j might be larger than 2  j , resulting in a negative value of the integral in (18a). Subregion 5 in Fig. 1(b) is one example where 1  5
C @ @ @

2  5 . Substituting (17) into (18) and solving the inner integral with respect to r, we get
2 A j  L 

Pe  j

@ 1 A j

2a b  R  a b  R  cR2 


b  3 E 2

2b  tan  3 E 2

1 F

b 

2cR 

HPI

d @
@ @ @

(19a)

where [12, eqns. (2.175.2) and (2.172)] have been used and
 L

4ac


b 


2 2  mxy  

4 mxx myy

4  myy sIi sin

 C

sQi cos  (19b) (19c) (19d)

0 mxy 3 cos


0 mxy 3 sin ! 2
2

mxx myy


 mxy 

mxx myy mxy

E  zz
!

Eg g
!

(19e) (19f)

E  zg

The symbol error probability of any coherent two-dimensional constellation in Rayleigh fading with channel estimation errors is simply a weighted sum of terms given by eqn. (19a) [2], Pe

jQ 1

w j Pe  j

(20)

where N is the total number of distinct erroneous subregions of the 2-D constellation and w j is the weighting coefcient [2]. The new single integral expression (19) is an exact solution to calculating the SER of coherent 2-D signaling in Rayleigh fading with imperfect channel estimation. Note that only one level of integration is required, in contrast to four levels of nested integration required by previous methods. As a special case of 2-D signalings, the symbol error probability of MPSK in Rayleigh fading with channel estimation errors can be obtained by assuming si 1 as Pe  MPSK

 L
M

2a1 sin2 9 M  b1  sin 9 M  sin 9 M    1 a1 sin2 9 M  b1  sin 9 M  sin 9 M  c sin2 9 M  2b1 

3E 2 1

b1 
  
3E 2 1

tan 

1 F

b1  sin 9 M  2c sin 9 M  G  sin 9 M  1



H 6

 L

2a1 sin2 9 M  b1  sin 9 M  sin 9 M      1 a1 sin2 9 M  b1  sin 9 M  sin 9 M  c sin2 9 M  2b1  1

3E 2

b1 


3E 2

tan 

1 F

b1  sin 9 M  2c sin 9 M  G   sin 9 M  1

 

H 6

(21a)

where a1

myy mxx


2 0 mxy 3


(21b)


b1  c 1

2 myy cos myy 4 mxx myy



0 mxy 3 cos

0 mxy 3 sin 

(21c) (21d)

2  mxy  

4 myy sin 0 mxy 3 cos



0 mxy 3 sin 

(21e)

Note that the two integrals on the right side of (21a) are equal when the second moment m xy is real, i.e., 0 mxy 3

0 and, hence, b1


b1  . Thus, when a channel estimation method has the property that

10

the cross-correlation between the received signal z and the channel estimate g is real, the probability of symbol error for MPSK is given by doubling the rst integral. As will be shown in the next section, this is the case for pilot symbol assisted modulation. It is worth pointing out that for MPSK, it is actually easier to work with the decision variable D rather than t , because the erroneous decision region is more conveniently described by the non-shifted original coordinates and the SER is given by Pe  MPSK
 

M M

fD rd d  drd d d
   
3E 2
M

 

fD rd d  drd d d

(22a) (22b)

f d  d d
M

f d  d d

 L
M

2 2
 5

b2 d  2


2b2 d  2

3E 2

tan 

1 bG 2

d  2
6

d d d d
6

 

 L

2 2

b2 d 
3E 2 2

2b2 d 
3E 2 2

tan 

1 bG 2

d  2

(22c)

where f d  is the marginal PDF of d , the phase difference between the received signal and the channel phase estimate, and b2 d 


2 0 mxy 3 cos d 0 mxy 3 sin d 

2

(22d)

4mxx myy 4 0 mxy 3 cos d 0 mxy 3 sin d 

(22e)

Another special case is the symbol error probability of M -ary rectangular QAM, which is often referred to as MQAM. This is readily obtained using (20) and (19), though the details are omitted for brevity. More generally, we have written a computer program that calculates these geometric parameters automatically for arbitrary 2-D constellations. III. APPLICATIONS TO CHANNEL ESTIMATION ERROR ANALYSIS This section applies the newly derived method to the performance analysis of 2-D signalings in the presence of three different kinds of channel estimation errors.

11

A. Static Channel Estimation Errors Two parameters, amplitude error tolerance and phase error tolerance, were dened in [2] to describe how sensitive a 2-D constellation is to carrier amplitude and phase tracking errors. These two parameters provide some information about the robustness of a modulation format, but the effect of channel estimation errors on the average SER of the modulation format is not known with accuracy. Here we investigate the performance of 2-D signaling in Rayleigh fading with constant channel estimation errors caused by the carrier amplitude and phase tracking components. Another scenario where static channel estimation error is relevant is when amplier nonlinearity is present in the system and the estimated amplier nonlinearity differs from the true amplier nonlinearity by some constant error. In all these cases, the estimated fading is related to the real fading by a constant factor. That is, g error and is the phase estimation error. Parameters q 1 and 0 correspond to the perfect channel estimation case. Therefore, the fading estimate g is also a zero-mean complex Gaussian R.V. correlated to the true fading g. Letting X

g q exp 1 j 2

where q is the amplitude estimation

z gsi n and Y

g q exp 1 j 2

, the second moments given signal si

transmitted are mxx

myy mxy

2Es  si  2 2N0 2Es q2 2Es si j e q

(23a) (23b) (23c)

The signal-to-noise ratio (SNR) per symbol is given by Es 9 N0 . The SER of arbitrary 2-D signaling in the presence of static channel estimation errors can be readily obtained from (19) and (23). The second moments are obtained from (23) with si 1 for MPSK. The probability of error for MPSK in Rayleigh fading with static channel estimation errors is then given by (21) and (23). B. Pilot Symbol Assisted Modulation In a PSAM system, pilot signals are periodically inserted into the data stream for every L symbols. Hence, a data frame of length L consists of a pilot symbol at its zero-th position and L
 

1 data 1 data

12

symbols at positions from 1 to L is obtained by g k p



1. The estimated channel fading at the pilot position of the k-th frame

zk  0 9 pk where pk is the pilot symbol of the k-th frame and zk  0 is the received signal at

this pilot position. The received signal at the l -th data symbol position of the current frame is given by zl

g l sl n l

(24)

where gl is the zero-mean complex Gaussian R.V. representing the multiplicative channel fading, s l is the l -th data symbol taking values from an M -ary signal set and nl is the additive Gaussian noise. Channel fading gl is estimated from K pilot symbols, that is, the previous K1 current pilot symbol and the subsequent K2 g l
UR SR

K


1  9 2T pilot symbols, the

K 9 2T pilot symbols as

kQ

K2

K1

k hk  l g p

kQ

K2

K1

hk  l V gk p

nk p pk W

(25) h l


where

RYX T

is the oor function, gk p is the actual channel fading at the k-th pilot position, and h k  l

kL  is the interpolator coefcient for g k p and is dependent on the current symbol position l . Proposed interpolators include the optimum Wiener interpolator [3], a Gaussian interpolator [13] and the simple sinc interpolator [4]. As an example, the sinc interpolator with a Hamming window is given by h n

w n  X sinc `

n La

K1 L
b

n b K2 L

(26a)

where w n

0 54


0 46 cos
V

2 R KL 2n 2 T KL 1 KL 1 W
 

(26b)

It can be seen from (25) that g l is also a zero-mean complex Gaussian R.V. because it is a linear combi2 nation of g p K1 g K p that are jointly Gaussian R.V.s. Let X

zl and Y

g l . To calculate the SER of

13

PSAM 2-D signaling, the only quantities needed from (19) are the second moments mxx

myy mxy Hl Ckn

2Es  sl  2 2N0 2N0 2 Hl CHT  Hl  l  pk  2

kQ

(27a) (27b) lT



 

K2

K1

2Es hk  l sl J0 2 fD  kL hK2  l ! 2Es J0 2 fD  k



(27c) (27d)

h

K1  l

n E  gk pg p

n T 

(27e)

where Ckn is the element with index k n  of the covariance matrix C for fadings at pilot symbols, f D is the maximum Doppler frequency and T is the symbol duration. Pilot symbol p k is assumed to have unit average energy, i.e.,  pk  2

1 in (27). The signal-to-noise ratio per symbol is given by Es 9 N0 . Since the

second moments myy and mxy are dependent on the symbol position l where l symbol error probability should be averaged over the L


1 L


1, the overall 1. Since

1 positions within a frame.

In the case of pilot symbol assisted MPSK, the second moments are given by (27) with s l MPSK in Rayleigh fading is thus given by two times the rst integral in (21a). C. Minimum Mean Square Error Estimation

mxy in (27c) is real, the two integrals in (21a) are equal. The probability of error for pilot symbol assisted

It is well known that the estimate and the estimation error are uncorrelated in minimum mean square error estimation and that the average estimation error power is the difference of the power of the variable to be estimated and the power of the estimate. Since the fading and its estimate are complex Gaussian R.V.s, MMSE estimate g and the estimation error e g


g are independent. Denote the mean power of

the estimation error by 22 e and the fading power by 2Es . The estimation error power is a measure of the quality of the estimation algorithm. Let X

z and Y

g . The second moments required by (19) in the

14

case of MMSE channel estimation are given by mxx

2Es  si  2 2N0 E  gsi n  g 2Es

 !

(28a) E e g  si g
!

mxy myy This leads to

myy si

(28b) (28c)

22 e

mxx


myy  si  2
C

(29a) (29b)

b c

0 myy
C

(29c)

Substituting (29) into (19), we have for MMSE estimation

MMSE Pe j

@ @

@ @

2 A j  L 

@ 1 A j

sin2 B 1  j  j   d @@ 2 2c  a sin2 B 1  j  j  cx2 @ j sin !

@ @  G

(30)

Eqn. (30) can be further simplied to a closed-form expression given by

MMSE Pe j

 2  j

1  j   2

1

cu j
'( '(gfh 

2 7 cu2 j a a tan B 2  j 1  j   cu2 j j

de

tan 

$&

tan 

$&

a tan j cu2 j

(31)

where u j

x j sin j
C

0 and [12, eqn. (2.562.1)] has been used. For MMSE MPSK, the symbol error
G  G

probability can be written in closed-form as

MMSE Pe  MPSK

M 1  M

cu
D

cu2 a

tan 

1 Fpi

a tan 2 cu M HqI

(32)

where u sin M , and a and c can be obtained from (29) and (28) with si

1.

Both (31) and (32) are new results. Substituting (29) into (17), we obtain an equivalent PDF expression in polar coordinates for the combined noise term to that given by [7, eqn. (22)]. Polar coordinates facilitate

15

the present analysis and lead to simplied and generalized results for arbitrary 2-D signaling. While [7, eqn. (22)] is valid only for MMSE channel estimation, (17) applies to any channel estimations that are jointly Gaussian with the actual channel fading. Note that our new closed-form SER expression (31) applies to arbitrary polygonal 2-D constellations with MMSE estimation in Rayleigh fading, compared to the single integral results given in [7] for SERs of 4-PSK, 8-PSK, 8-AMPM, 16-PSK and 16-QAM. IV. NUMERICAL EVALUATIONS We present in this section some numerical results for various 8-ary and 16-ary constellations as applications of the theoretical analysis presented in Sections II and III. The 8-ary signalings considered are 8PSK, the 8-ary rectangular set, the 8-ary max-density, the 8-ary triangular set, (4,4) and (1,7). The 16-ary signalings considered are the 16-ary hexagonal set, 16 rectangular QAM, the 16-ary triangular set, (4,12), the 16-ary max-density, (5,11), 16 star-QAM, rotated (8,8) and (1,5,10). The signal space diagrams for these constellations can be found in [2] and are not presented here due to space limitations. Figs. 2-4 show the SERs of the 8-ary and 16-ary signal sets as functions of the average signal-to-noise ratio per bit, in the presence of static channel estimation errors. The SNR per bit

Eb N0 Eb N0 ,

is related to

Es N0

by a factor of

log2 M 


1.

The derived single integral (19) with nite integration limits is easy to evaluate numerically

and generates results with high accuracy. Fig. 2 demonstrates the effect of constant channel estimation errors on the SER performance of 8-ary signal sets in Rayleigh fading. The performance of the (4,4) constellation will depend on the ring ratio employed. The ring ratio of (4,4) is set at 2.4544, the optimized value for SNR 20 dB. It is evident that the relative performances of these constellations with channel estimation errors are distinct from those with perfect channel state information as given in [2]. Given constant amplitude error q 2 dB and phase error 10 r , (4,4) and (1,7) have better error performance than 8PSK and the 8-ary rectangular set in a Rayleigh fading environment. The (1,7) set saves about 2.04 dB power over 8PSK at SER 1e (4,4) saves around 1.96 dB over 8PSK as shown in Fig. 2.


3 and

The performances of some 16-ary constellations are presented in Figs. 3 and 4 for the ring ratios speci-

16

ed in the gure captions. The performance differences among 16-ary signal sets in Figs. 3 and 4 are more pronounced than those of 8-ary signal sets in the presence of the same static channel estimation errors, as expected. Circular constellations with two rings such as 16 star-QAM, rotated (8,8), (5,11) and (1,5,10) are clearly good choices that are robust to constant channel estimation errors as shown in Figs. 3 and 4. The 16-ary hexagonal set, 16 rectangular-QAM, the 16-ary max-density and the 16-ary triangular set no longer yield acceptable performance in the presence of the amplitude error q


2 dB and the phase error

10 r . Sixteen star-QAM and rotated (8,8) signaling have comparable error performances. Comparing Fig. 3 with Fig. 4, it is seen that under-estimating the channel amplitude has less adverse effect on the SER than over-estimating the channel amplitude with the same  q  dB. Some simulation results are plotted in Figs. 2 and 4 to verify the analytical SER results. Figs. 5-9 present SER results for pilot symbol assisted modulation schemes which introduce dynamic channel estimation errors to receiver detection. As described in Section III-B, we use a sinc interpolator with Hamming windowing in our numerical evaluations. Note that the general SER expression (19) is applicable to any linear interpolator and our study shows that Hamming windowing produces better performance than rectangular windowing. Fig. 5 depicts the SERs of pilot symbol assisted 16 rectangular-QAM and 16 star-QAM as a function of SNR in Rayleigh fading. Sixteen rectangular-QAM slightly outperforms 16 star-QAM for the perfect channel state information case, and the benet is greater at higher SNR. However, it is clear from Fig. 5 that when dynamic channel estimation errors inherent in PSAM schemes are present, the performance degradation of 16 rectangular-QAM is quite severe. The SNR difference by numerical evaluation for star-QAM with and without channel estimation error at SER=1e


is about 2.4 dB, while for rectangular-QAM, the SNR difference is about 3 dB. This shows that star-QAM is more robust to channel estimation errors than rectangular-QAM. Pilot symbol assisted 16 star-QAM has a slightly better performance than pilot symbol assisted 16 rectangular-QAM, and at very high SNR both constellations exhibit error oors. The error oor of 16 star-QAM, however, is less than that of 16 rectangular-QAM. Simulation results agree well with our theoretical analysis. It is also observed in our study that performance differences between the two constellations are more pronounced at higher fading

17

rates when channel estimation errors increase. The choice of two parameters, frame length L and interpolation order K , depends on a number of factors. The larger the frame length L, the less power loss incurred from the pilots inserted in the data stream. However, L is upper bounded by the Nyquist Sampling Theorem at L
b
1 2 fD T .

For example, L

should be less than 16 for f D T

0 03. Therefore we choose L

15. We found that the PSAM system

would not work with larger values of L, and smaller L does not lead to much performance gain. Once L is determined, the SER performance of 16 rectangular-QAM as a function of interpolation order K is plotted in Fig. 6. Parameter K determines the buffer size and the detection delay and should be minimized without sacricing the error performance. As can be seen from Fig. 6, the smaller the average SNR of the channel, the smaller the value of K that is sufcient. It is obvious that choosing K

30 works well for SNR values

less than 40 dB, but is not sufcient for SNR = 50 dB, where K

34 reduces error oors noticeably. It

is observed in our study that K depends on the channel fading rate, the SNR and the frame size L. It does not seem to depend strongly on the signal constellation used. The ring ratio of pilot symbol assisted 16 star-QAM requires optimization for minimum SER. Note that in slow Rayleigh fading with perfect channel state information, the optimum ring ratios are a function of SNR and approach an asymptotic value as SNR gets very large [2]. The asymptotic optimum ring ratio is 1.952 in Rayleigh fading. Fig. 7 shows the optimum ring ratio of PSAM 16 star-QAM as a function of SNR for fD T

0 03, L 15, K

30 and 34. Obviously the optimum ring ratios no longer approach an

asymptotic value and are quite different from those of the perfect channel knowledge case. The optimum ring ratio of 16 star-QAM is chosen at 2.18 in Fig. 5. Fig. 8 shows that pilot symbol assisted (4,4) achieves the best SER performance among 8-ary signal sets with fD T

0 03, L

15, and K

30 in Rayleigh fading. The ring ratio used for (4,4) is 2.4065,

optimized for SNR 20 dB. There is about 1 6 dB power savings for PSAM (4,4) over PSAM 8PSK at SER 1e


3, which is rather signicant when it comes to system design for high speed data. Error oors

are observed for all signalings at high SNR. The SERs of pilot symbol assisted 16-ary signal sets in Rayleigh fading are plotted in Fig. 9 with

18

fD T

0 03, L 15, and K

30. Fig. 9 shows that 16 star-QAM has very comparable performance to the

rotated (8,8) set and achieves the lowest SER among the ve signal sets in the medium to high SNR range. Although not plotted here, it is also found in our study that the constellation (5,11) curve is slightly above that of the 16-ary max-density, and (1,5,10) almost overlaps with rotated (8,8). The ring ratios used for 16 star-QAM and rotated (8,8) in Fig. 9 are 2.1674 and 2.1039, respectively. Fig. 10 plots the SER curves of 16 rectangular-QAM, 16 star-QAM and QPSK in Rayleigh fading when MMSE estimation errors are present. Suppose the variance of the estimation error is 5 dB, 16 star-QAM suffers 4.86 dB power loss while 16 rectangular-QAM loses 5.8 dB power for SER at 1e


3, compared

to their corresponding perfect coherent detection. In addition, for symbol error probabilities smaller than 1e


2, 16 star-QAM saves about 0.6 dB power over 16 rectangular-QAM with 2 e

5 dB. The QPSK

curves obtained agree with the results in [8, Fig. 6], except that the horizontal axis in Fig. 10 is SNR per bit whereas it is SNR per symbol in [8, Fig. 6]. V. CONCLUSION The impact of channel estimation errors on the performance of coherent 2-D signaling has been investigated in this paper. A new, simple, general method of analysis has been developed for performance evaluation of arbitrary polygonal 2-D signaling in Rayleigh fading with imperfect channel estimation. The symbol error probability of an arbitrary polygonal 2-D constellation is given by a single integral with nite integration interval and is well-suited to numerical evaluation. The practical effects of channel estimation errors on signaling performance have been examined by considering static channel estimation errors, pilot symbol assisted modulation and minimum mean square error estimation. The SER expression of arbitrary 2-D signaling with MMSE estimation was further simplied to a closed form. The symbol error probabilities of six 8-ary signal sets and nine 16-ary signal sets have been numerically evaluated and plotted under different kinds of channel estimation errors. Pilot symbol assisted (4,4) displays signicant power advantage over conventionally used PSAM 8PSK. Pilot symbol assisted 16 star-QAM has been shown to be more immune to channel estimation errors than PSAM 16 rectangular-QAM and its performance

19

is comparable to PSAM rotated (8,8). It has also been demonstrated that parameters crucial to channel estimation schemes and ring ratios of circular constellations can be easily and accurately determined using the analytical results presented. ACKNOWLEDGEMENT The rst author wishes to thank Mr. Lei Xiao for his assistance in simulations. R EFERENCES
[1] X. Dong, N.C. Beaulieu and P.H. Wittke, Error Probabilities of Two-Dimensional M-ary Signaling in Fading, IEEE Trans. Commun.,vol. 47, pp. 352-355, Mar. 1999. [2] X. Dong, N.C. Beaulieu and P.H. Wittke, Signal constellations for fading channels, IEEE Trans. Commun., vol. 47, pp. 703-714, May 1999. [3] J.K. Cavers, An Analysis of Pilot Symbol Assisted Modulation for Rayleigh Fading Channels, IEEE Trans. Veh. Technol., vol. 40, pp. 686-693, Nov. 1991. [4] Y.S. Kim, C.J. Kim, G.Y. Jeong, Y.J. Bang, H.K. Park and S.S. Choi, New Rayleigh Fading Channel Estimator Based on PSAM Channel Sounding Technique, Proc. IEEE Int. Conf. Communications (ICC97), pp. 1518-1520, June 1997. [5] X. Tang, M.S. Alouini and A. Goldsmith, Effect of channel estimation error on M-QAM BER performance in Rayleigh fading, IEEE Trans. Commun., vol. 47, pp. 1856-1864, Dec. 1999. [6] K. Yu, J. Evans and I. Collings, Performance Analysis of Pilot Symbol Aided QAM for Rayleigh Fading Channels, Proc. IEEE Int. Conf. Communications (ICC02), Apr. 2002. [7] A. Aghamohammadi and H. Meyr, On the Error Probability of Linearly Modulated Signals on Rayleigh Frequency-Flat Fading Channels, IEEE Trans. Commun., vol. 38, pp. 1966-1970, Nov. 1990. [8] M.G. Shayesteh and A. Aghamohammadi, On the Error Probability of Linearly Modulated Signals on Frequency-Flat Ricean, Rayleigh, and AWGN Channels, IEEE Trans. Commun., vol. 43, pp. 1454-1466, Feb./Mar./Apr. 1995. [9] S.K. Wilson and J.M. Ciof, Probability Density Functions for Analyzing Multi-Amplitude Constellations in Rayleigh and Ricean Channels, IEEE Trans. Commun., vol. 47, pp. 380-386, Mar. 1999. [10] R.A. Wooding, The Multivariate Distribution of Complex Normal Variables, Biometrika, vol. 43, pp. 212-215, June 1956. [11] J.W. Craig, A New Simple and Exact Result for Calculating the Probability of Error for Two-Dimensional Signal Constellations, Proc. IEEE Milit. Commun. Conf. (MILCOM91), Boston, MA, pp. 571-575, 1991. [12] I.S. Gradshteyn and I.M. Ryzhik, A. Jeffrey (editor), Table of Integrals, Series, and Products. Fifth Edition, San Diego, CA: Academic Press, 1994. [13] S. Sampei and T. Sunaga, Rayleigh fading compensation for QAM in land mobile radio communications, IEEE Trans. Veh. Technol., vol. 42, pp. 137-147, May 1993.

20

C 2,1

Si
3

1,1

x1

1
A

R = |OE|
D

4 (a)

x 1 = |OA|

5 5
1,5 2,5 2,7 O 1,7

x5

Si x7

7
G

x 5 = |OF| x 7 = |OG|

(b)

Fig. 1. Decision regions and geometric parameters of a (a) closed region and (b) open region.

21

10

10

8PSK 8PSK simulation 8 Rectangular 8 Maxdensity 8 Triangular (4,4) (1,7)

Symbol Error Probability

10

10

q=2 dB, =10 10

4

10

10

15

20

25

30

35

40

45

50

Average Eb/N0 (dB)

Fig. 2. Average SERs of 8-ary signal sets in Rayleigh fading with amplitude error q Q 2 dB and phase error Q 10 s .

22

10

10

Symbol Error Probability

10

q=2 dB, =10

10

10

16 rect.QAM Triangular Hexagonal (4,12) Maxdensity (5,11) 16 starQAM Rot. (8,8) (1,5,10)

10

10

15

20

25

30

35

40

45

50

Average Eb/N0 (dB)

Fig. 3. Average SERs of 16-ary signal sets in Rayleigh fading with amplitude error q Q 2 dB and phase error Q 10 s . The ring ratios used for (1,5,10), rotated (8,8), 16 star-QAM, (5,11) and (4,12) are 2.5933, 2.2893, 2.2947, 2.8487 and 3.1764, respectively.

23

10

10

Symbol Error Probability

10

q=2 dB, =10o

10

10

16 Hexagonal 16 rect.QAM 16 rect.QAM simulation 16 Maxdensity 16 Triangular (4,12) Rot. (8,8) 16 starQAM 16 starQAM simulation (5,11) (1,5,10)

10

10

15

20 25 30 Average Eb/N0 (dB)

35

40

45

50

Fig. 4.

2 dB and phase error Q 10 s . The ring ratios  used for (1,5,10), rotated (8,8), 16 star-QAM, (5,11) and (4,12) are 2.8939, 2.6987, 2.6846, 3.2253 and 3.497, respectively. Average SERs of 16-ary signal sets in Rayleigh fading with amplitude error q Q

24

10

10

16 rect.QAM, simulation 16 starQAM, simulation 16 rect.QAM, theory 16 starQAM, theory 16 rect.QAM, perfect 16 starQAM, perfect

Symbol Error Probability

10

10

L=15, K=30, fDT=0.03 10

4

10

10

20

30 Average Eb/N0 (dB)

40

50

60

Fig. 5. Average SERs of PSAM 16 rectangular-QAM and 16 star-QAM in Rayleigh fading with f D T Q 0 t 03  L Q 15  K Q 30 t

25

10

10

Symbol Error Probability

10

16 rect.QAM f T=0.03 L=15 10

3

10

10

SNR=10 dB SNR=20 dB SNR=30 dB SNR=40 dB SNR=50 dB 5 10 15 20 K 25 30 35 40

Fig. 6. Average SERs of PSAM 16 rectangular-QAM as a function of K in Rayleigh fading with f D T Q 0 t 03  L Q 15 t

26

2.5

K=30 K=34

2.45 16 starQAM L=15 fDT=0.03

2.4

Optimum Ring Ratio

2.35

2.3

2.25

2.2

2.15

2.1

10

15

20

25

30

35

40

Average Eb/N0 (dB)

Fig. 7. Optimum ring ratios of PSAM 16 star-QAM in Rayleigh fading with f D T Q 0 t 03  L Q 15  K Q 30 and 34 t

27

10

10

8PSK 8PSK simulation 8 Rectangular (1,7) 8 Maxdensity 8 Triangular (4,4)

Symbol Error Probability

10

10

L=15, K=30, fDT=0.03 10

4

10

10

20

30 Average Eb/N0 (dB)

40

50

60

Fig. 8. Average SERs of PSAM 8-ary signal sets in Rayleigh fading with f D T Q 0 t 03  L Q 15  K Q 30 t

28

10

10

Hexagonal 16 rect.QAM Maxdensity Rot. (8,8) 16 starQAM

Symbol Error Probability

10

10

L=15, K=30, f T=0.03

D

10

10

10

20

30 Average Eb/N0 (dB)

40

50

60

Fig. 9. Average SERs of PSAM 16-ary signal sets in Rayleigh fading with f D T Q 0 t 03  L Q 15  K Q 30 t

29

10

10

16 rect.QAM, MSE=5 dB 16 starQAM, MSE=5 dB 16 starQAM, perfect 16 rect.QAM, perfect QPSK, MSE=1/3 [8] QPSK, perfect

Symbol Error Probability

10

10

10

10

10

15

20 25 30 Average Eb/N0 (dB)

35

40

45

50

Fig. 10. Average SERs of 16 star-QAM and 16 rectangular-QAM in Rayleigh fading with MMSE estimation and 2 e Q 5 dB, and the SER of QPSK with 2 e Q 1 E 3.