Escolar Documentos
Profissional Documentos
Cultura Documentos
De
oding of Turbo{Codes .
1
The following problems related to the
onstru
tion and analysis of turbo{
odes are
onsidered: the asymptoti
behavior of the interleavers (permutors), the asymptoti
be-
havior of the minimum distan
e, and also some examples of the pra
ti
al appli
ation of
the developed methods to
on
rete turbo{
odes.
x1 Introdu tion
Turbo{
odes, also
alled parallel
on
atenated
odes, were introdu
ed by a group of fren
h
authors in 1993 [1℄. They
an be
onsidered as a parti
ular
ase of low{density parity{
he
k
odes [2℄. The main advantage of these
odes is that they
an be de
oded iteratively.
While the
omplexity of iterative de
oding is
onsiderably lower than the
omplexity of
maximum likelihood de
oding (ML), the reliability is signi
antly higher than for ML
de
odable
odes with the same
omplexity.
There exist two dierent approa
hes to the analysis of low{density
odes. In the rst
approa
h random ensembles of
odes are
onsidered and
ode
hara
teristi
s averaged over
the whole ensemble are estimated. This approa
h is produ
tive for the analysis of general
hara
teristi
s of the
odes (for example minimum distan
e, spe
trum) and maximum like-
lihood de
oding. The se
ond approa
h is to analyze spe
ially
hosen
odes. The existen
e
of these
odes in the ensemble is being proved using spe
ial
onstru
tive methods. From
our point of view this way is preferable for the analysis of iterative de
oding of
odes. The
two approa
hes were introdu
ed in the work of Gallager [2℄ and developed, parti
ularly,
in the papers of Zyablov and Pinsker [3℄, Margulis [4℄,[5℄,[6℄, Tanner [7℄, Benedetto and
Montorsi [8℄ and other re
ently published works.
1 This work was supported in part by Swedish Resear
h Coun
il for Engineering S
ien
es (Grant 98{216).
1
In this arti
le we adhere to the se
ond approa
h and analyze the asymptoti
properties
of iterative de
oding of binary turbo{
odes. The de
oding
omprises the iterative
al
ula-
tion of the a posteriori probabilities of information symbols. The \
orre
t"
al
ulation of
these a posteriori probabilities puts some
onstraints to the stru
ture of the turbo{en
oder,
parti
ularly to the
onstru
tion of the permutor (interleaver)2. The main results of this
work are a proof of the existen
e of a permutor, providing \
orre
t"
al
ulation of the a
posteriori probabilities of the symbols and a proof of the existen
e of turbo{
odes with a
minimum distan
e growing logarithmi
ally with the length of the
ode.
Binary turbo{
odes with
omponent en
oders having memory equal to one are studied
more in detail. Although these
odes have worse
hara
teristi
s than turbo{
odes with
omponent en
oders with greater memory their stru
ture allows a simple mathemati
al
analysis. The authors are going to
arry out the analysis of more
ompli
ated turbo{
odes
in a dierent arti
le [9℄ using the results of this paper.
The ideas of the proof of the existen
e of asymptoti
ally \good" permutors were used
for the pra
ti
al
onstru
tion of permutors with relatively short length. In the nal part
of the arti
le we will demonstrate that turbo{
odes with spe
ially
onstru
ted permutors
perform signi
antly better than turbo{
odes with randomly
hosen permutors.
A
lassi
al example of a rate 1=3 turbo{en
oder is
onsidered in Example 3 of the paper [10℄.
The en
oder uses systemati
re
ursive
onvolutional (7; 5) en
oders as
omponent en
oders
and a standard interleaver
onsisting of two tables as a permutor. In this paper we
onsider
a
lass of rate 1=3 binary turbo{en
oders
ontaining the turbo{en
oder des
ribed above
2 The term\interleaver" is usually used in the te
hni
al literature to des
ribe a devi
e
onsisting of a
table whi
h is lled
olumnwise by input symbols and read rowwise. We will denote by \permutor" a
general devi
e that
hanges the order of the symbols in the input information sequen
e. The
lass of these
devi
es
ontains usual interleavers as parti
ular
ase. Before, we used the term \s
rambler" but sin
e
this term determines other devi
es in
ommuni
ation theory and
ryptography we de
ided to
hange it,
following the advi
e of J. Massey.
2
as a parti
ular
ase. The generalization is that any systemati
re
ursive rate 1=2 memory
m
onvolutional en
oders
an serve as
omponent en
oders and spe
ial permutors are
used instead of the standard interleaver. The
onstru
tion of these permutors is dis
ussed
in the following two paragraphs.
Mathemati
ally, a blo
k permutor
an be dened as a square permutation matrix
of size N N , having a single one in ea
h row and in ea
h
olumn and zeros in the
other positions. The task of the permutor is to
hange the order of the symbols in the
input information sequen
e of length N : if a binary ve
tor u = u0; u1; : : : ; u 1 enters the
N
permutor then the ve
tor u = u; u = u00; u01; : : : ; u0 1 appears at it's output.
0 0
N
The binary sequen
es u and u enter two parallel binary
omponent en
oders whi
h
0
are systemati
re
ursive rate 1=2 memory m
onvolutional en
oders. The output of the
rst en
oder is the binary
ode sequen
e v = (v0(1) ; v1(1) ; : : : ; v(1) 1 ), the output of the
(1)
N
se
ond the binary sequen
e v = (v0(2) ; v1(2) ; : : : ; v(2) 1 ). The sequen
es u, v and v
(2)
N
(1) (2)
state, whi
h usually is the zero state. Additionally to u, v and v these tails and the
(1) (2)
orresponding
ode symbols are then transmitted over the
hannel. This
auses a de
rease
of the transmission rate, but sin
e usually N m
the de
rease is not signi
ant. The
se
ond way is to use a \tail{biting" method. In this
ase the starting states of the en
oders
are
hosen su
h that the input sequen
es u and u bring the en
oders to these starting
0
states.
For memory m
> 1
omponent en
oders a ne
essary and suÆ
ient
ondition for the
possibility to
onstru
t a \tail{biting"
ode is that the polynomial 1+ D and the feedba
k
N
polynomial of the en
oders are relatively prime [11℄. In this
ase the transmission rate of the
orresponding turbo{
ode is 1=3 (bits/
hannel symbol). For memory m
= 1
omponent
en
oders a ne
essary and suÆ
ient
ondition to build a tail{biting
ode is that the input
information sequen
e of ea
h
omponent en
oder has even Hamming weight. This
an be
3
a
hieved, for example, by adding a parity{
he
k symbol to the input information sequen
e.
Later on we will assume that input sequen
es of memory one en
oders
onsist of N 1
information and one parity{
he
k symbol. The transmission rate is equal to (1 1=N )=3.
In this paper we will follow the se
ond way and
onsider tail{biting turbo{
odes.
We investigate the appli
ation of these
odes to the transmission over the binary sym-
metri
hannel (BSC) and the additive white Gaussian noise (AWGN)
hannel. Let
r = (r0; r1; : : : ; r 1), r(1) = (r0(1), r1(1) , : : : , r(1) 1 ), and r(2) = (r0(2) , r1(2) , : : : , r(2) 1)
N N N
denote the re
eived information sequen
e, the parity{
he
k sequen
e of the rst en
oder
and the parity{
he
k sequen
e of the se
ond en
oder, respe
tively.
In order to des
ribe the de
oding pro
edure we introdu
e the Lee metri
on the set N
= f0; 1; : : : ; N 1g of indi
es of symbols u : dL(i; j ) = min(ji j j; N ji j j), i; j 2 N .
n
The de
oder operates in the following way. First, using the sequen
es r and r(1) it
al
ulates the a posteriori probabilities (APP) of the symbols from the information sequen
e
u. This is done by a symbolwise APP
omponent de
oder that is based on a forward
ba
kward algorithm, known as the BCJR algorithm [12℄. We have
onsidered a modi
ation
of the BCJR algorithm, operating on a sliding window of size 2 + 1. In this
ase, for
the
al
ulation of the a posteriori probability for symbol u , the rst
omponent de
oder
n
operates with r ; r(1) , the 2 re
eived symbols of the sequen
e r whi
h are
losest (here
n n
and further on we
onsider the Lee metri
) to r and the 2 re
eived symbols of the
n
sequen
e r(1) ,
losest to r(1) . In the se
ond de
oding iteration the se
ond
omponent
n
now use of the newly al ulated a posteriori probabilities of the 2 symbols losest to u0 0 , n
symbol r(2)0 , the 2 symbols of the sequen
e r(2)
losest to r(2)0 and symbol r . Here n0
n n n
denotes the index of the symbol u in the sequen
e u0. The additional information from
n
the previous step is not used about the symbol u0 0 itself, in order to obtain independent
n
4
teriori probabilities of the symbols from the information sequen
e u, using the a posteriori
probabilities
al
ulated in the previous iteration and, as before, the 2 symbols of the
re
eived sequen
e r(1) ,
losest to r(1) . The following iterations, up to I 1, are
arried out
n
analogously. In the last iteration, I , the
omponent en
oder uses both the 2 symbols of
the sequen
e u,
losest to u and the 2 symbols of the sequen
e u0 ,
losest to u0 0 and
n n
The BCJR algorithm minimizes the bit error probability Pb of
onvolutional
odes
and is optimal in this sense. It
an be shown, that if ! 1 the performan
e of the
modied BCJR de
oder, operating on a window of size 2+1 approa
hes the performan
e
of the standard BCJR de
oder. Unfortunately, the BCJR algorithm
an not be analyzed
analyti
ally and should be investigated numeri
ally by Monte{Carlo methods. For the
purpose of analysis we introdu
ed a suboptimal APP de
oding algorithm for
onvolutional
odes,
alled \max{path" algorithm. It
al
ulates the probability of the most probable
path in the {trun
ated trellis
orresponding to u = 0 and the probability of the most
n
probable path in the {trun
ated trellis
orresponding to u = 1. The ratio of these two
n
BCJR algorithm.
To des
ribe the de
oding of symbol u it is
onvenient to introdu
e a tree{like graph,
n
{graph (Figure 1), analogous to the one
onsidered in [13℄. Ea
h node within an even
level of the tree represents one of the information symbols and ea
h node within an odd
level is asso
iated with one of the
omponent
odes, dening the relationship between the
information symbols
onne
ted to this node.
We will
all the set of nodes in the even levels of the tree \(u ; ){
lan". The root node
n
u (zero level),
orresponding to the symbol we started to build the tree from, we denote
n
by \
lan head". From this node arise two edges leading to the two nodes asso
iated with
the rst and the se
ond
omponent
ode. These two nodes of the rst level we will
all
the
lan head's \families" or \marriages" of the rst and se
ond type, respe
tively. Ea
h
of these two nodes is
onne
ted to the 2 nodes of the se
ond level that represent the 2
5
information symbols
losest to the
lan head u in the rst and the se
ond
omponent
ode,
n
respe
tively. These nodes of the se
ond level we will
all \
hildren" or \dire
t des
endants"
of the
lan head.
Ea
h dire
t des
endant of the
lan head has one family, i.e. ea
h node of the se
ond
level is
onne
ted with one node in the third level,
orresponding to the other
omponent
ode that
ontains this des
endant. If one parti
ular symbol belongs to a family of the rst
type as a des
endant then it is the head of a family of the se
ond type and vi
e versa. It
has 2
hildren in the fourth level. They are des
endants of the
lan head u in the se
ond
n
generation and so on. We will refer to the set of nodes in the 2lth level as lth generation.
A (u ; ){
lan is
alled \nondegenerated" up to the lth generation if all the symbol
n
nodes in the tree up to the lth generation are dierent. We
all a
lan degenerated in the
lth generation, if it is nondegenerated up to the (l 1)th generation and there exist two
nodes in the lth and kth (for some k l) generation asso
iated with the same information
symbol.
A
ode is
alled \(l0 ; ){nondegenerated", if the
lans of all
ode symbols are nonde-
generated up to the l0th generation.
We will mark all symbol nodes in the tree by zeros and ones depending on the value
of the information symbols
orresponding to the nodes. To simplify referen
es we will,
a
ording to [13℄,
all nodes marked by one \male nodes" and nodes marked by zero
\female nodes". By agreement, the surname (family name)
an be inherited by male
individuals only. If the head of a family is a man and all his des
endants are women,
then we talk about \bran
h extin
tion". If there are no inheritors in the lth generation,
l 2 f1; 2; : : : ; l0 g, we will talk about \family name extin
tion".
An upper bound on the nondegeneration level l0 of a
ode is presented by the next
simple theorem.
Theorem 1 For any (l0 ; ){nondegenerated turbo{
ode with permutor size N the nonde-
generation level l0 satises
l0 <
log2 N=2 : (1)
log2 (2)
6
Proof: Let us
onsider the nontrivial
ase N > 2. Then there are exa
tly 2
hildren in ea
h family. Let us
onsider one arbitrary
lan. Sin
e the
lan head has 2
hildren in ea
h of it's two families, the rst generation
ontains 4 symbols. Analogously,
the se
ond generation
ontains 2(2)2 symbols and so on. The l0 th generation
ontains
2(2) 0 symbols. For an (l0; ){nondegenerated
ode, all symbols in the
lan up to the
l
sequen e u. Let us introdu e a fun tion (i) = j labeling the index of the symbol u 2 u i
0 i ; j N 1, k = 1; : : : ; l, su
h that all i
k k k are dierent and j = (i ); k = 1; : : : ; l,
k k
Consider a sequen
e of distan
es (in Lee metri
): d1 = dL(i1 ; i2), d2 = dL(j2 ; j3), d3 =
dL (i3 ; i4 ), : : :, d = dL(j ; j1 ).
l l
Denition 2 Assume 9k1 ; k2 2 f1; : : : ; lg su
h that 1 dk1 ; dk2 2, and that 1 dk
, 8k 2 f1; : : : ; lg n fk1; k2g. Then is
alled a rst type {
y
le of length l.
Denition 3 Assume 9k1 2 f1; : : : ; lg su
h that 2 < dk1 3, and that 1 d ,
k
Lemma 1 If the permutation matrix does not
ontain rst type {
y
les of length less
than 2l0 + 1 and does not
ontain se
ond type {
y
les of length less than l0 then the
orresponding turbo{
ode is (l0 ; ){nondegenerated.
Denition 5 Assume 9k1 ; k2 2 f1; : : : ; l 1g, su
h that 1 dk1 ; dk2 2, and that
1 dk , 8k 2 f1; : : : ; l 1g n fk1; k2g. Then the sequen
e is
alled a rst type
{half{
y
le of length l. The positions (i ; j ); k = 1; : : : ; l 1 of the matrix are
alled
k k
verti
es and position (i ; j1 ) is
alled the free vertex of this {half{
y
le.
l
8
First we sele
t some position in the rst row of the matrix
ontaining A and
hange
it to 1. We pla
e a 0 in front of ea
h symbol in the matrix that is in the same row or
olumn as that 1. Then we
hange A to U in the positions of the matrix that be
ame
una
eptable. We
ontinue with the se
ond row of the matrix
hanging one of the symbols
A (but not 0A) to 1, pla
ing a 0 in front of ea
h symbol in the matrix that is in the same
row or
olumn as that 1, and
hanging A to U when ne
essary. We repeat this pro
edure
until we pla
ed a 1 in ea
h row of the matrix. If at some point in this pro
ess a row is
en
ountered, say the kth, in whi
h no position
ontains an A without an a
ompanying 0,
we do a spe
ial \emergen
y pro
edure".
Let
be a
olumn in whi
h the kth row
ontains a U ( = U ). Find a row i0 < k
k k;
k
1, 0 to 1, 0 0 to 0A, and
hange the symbols A, 0A, U and 0U in the whole matrix
k;
i i ;
i
16l03(2)2 0 N
l
; (3)
it is possible to perform the emergen
y pro
edure without forming prohibited
y
les.
4 + 3 log2 l0 + 2l0 log2 (2) log2 N < 4 + 3 log2(l0 + 1) + 2(l0 + 1) log2 (2) : (5)
A
ording to Lemma 2 there exists an (l0; ){nondegenerated
ode for this l0. Let us
further assume, that Theorem 2 is wrong and for ea
h (l0; ){nondegenerated
ode the
inequality
l0 + 1 <
1 (log N 3 log (log N ) 4) (6)
2 log (2) 2 2
2 2
9
is satised. Combining (5) and (6) we get
log2 N 3 log2(l0 + 1) 4 < log2 N 3 log2 (log2 N ) 4 ; (7)
2 log2(2) 2 log2(2)
and
onsequently
log2(l0 + 1) > log2 (log2 N ) : (8)
The last inequality is in
ontradi
tion with (6). This proves the theorem.
In the next paragraph we will analyze (l0 ; ){nondegenerated turbo{
odes with memory
one
omponent en
oders,
alled in short \memory one turbo{
odes".
Consider an (l0; ){nondegenerated memory one turbo{
ode with identi
al systemati
on-
volutional
omponent en
oders whose generator matri
es, in polynomial form, are equal
to (1 1=(1 + D)). The a
tive
olumn distan
e a [14℄ of this
ode is equal to the row
j
Let us study the asymptoti
minimum distan
e behavior of the
ode if N ! 1. Assume
N 4 and
hoose = b(log2 N )=2
, where b
denotes the integer part of a number.
Consider an arbitrary nonzero binary information sequen
e u at the input of the en
oder,
and an arbitrary nonzero symbol u(0) out of the sequen
e u. Let us
onstru
t an (u(0); ){
lan for this symbol. It will be nondegenerated up to the l0th generation a
ording to the
hoi
e of the turbo{
ode.
First we
onsider the
ase where family name extin
tion does not o
ur up to the l0 th
generation. Then there is at least one one in ea
h generation of the
lan. If the
onsidered
re
ursive systemati
en
oder is in the zero state an in
oming one produ
es at least one
parity{
he
k symbol. This implies that the weight of the
orresponding
ode sequen
e,
w(0) , is bounded from below by the inequality
w(0) 2l0
log2 N 5 4 (9)
log2 (log2 N ) log2(log2 N ) ;
10
whi
h follows from (4). Inequality (9) implies
log2 N
w(0)
log2 (log2 N ) 9 : (10)
Consider now the
ase when family name extin
tion o
urs in the lth generation, where
l l0 . Then there exists a nonzero symbol in the (l 1)th generation su
h that all it's
des
endants are zeros. This symbol, when entering the
omponent en
oder, produ
es a
burst of nonzero
ode symbols having weight at least + 1 a
ording to the formula for
the
olumn distan
e of the
ode. Hen
e, in this
ase the following inequality is satised
w 2+=2+
(0) log2 N log N
> 2 : (11)
2 2
It follows from (10) and (11) that for N 4 there exists a memory one turbo{
ode
whose minimum distan
e is bounded from below by the inequality
log2 N
dmin
log2(log2 N ) 9 : (12)
We will now prove that for any memory one turbo{
ode with permutor size N the
inequality
dmin 6 log2 N + 4 (13)
is satised.
Choose = 2. It follows from Theorem 1 that the nondegeneration level of a turbo{
ode with permutor size N satises
l0 log2 N 1 : (14)
Sin
e the turbo{
ode is not (l0 + 1; 2){nondegenerated there exists a rst type {
y
le of
length l 2l0 + 2 or a se
ond type {
y
le of length l l0 in the permutor matrix .
Assume = i1 ; i2; j2 ; j3; i3; i4 ; : : : ; j ; j1 is this
y
le and the
orresponding distan
es (in
l
Lee metri
) are fd1 = dL(i1 ; i2), d2 = dL(j2 ; j3), d3 = dL(i3 ; i4), : : :, d = dL(j ; j1 )g.
l l
symbols are zeros. The ones in the permuted sequen
e u = u will be pla
ed in the
0
11
positions j1; j2 ; : : : ; j . It follows from the formula for the a
tive
olumn distan
e, that the
l
sequen e u. To simplify the analysis we onsider the ase when the de ision u^ is made n
using the set of des
endants from only one marriage of u in the (u ; ){
lan. This in-
n n
reases the bound for the de oding error probability. Let us denote by V (u ) the set of n
des
endants of the head of the (u ; ){
lan up to the l0th generation and
ode symbols
n
by V0(u ) and
odewords with u = 1 by V1(u ). Let P (R(u )jV (u )) be the
onditional
n n n n n
probability to re
eive the word R(u )
onditioned that the word V (u ) was sent. It
an
n n
be shown that if the de
oder uses the \max{path" algorithm in ea
h iteration step then
iterative de
oding with I = l0 iterations is equivalent to the determination of two maxima:
12
P (0) = max P (R(u )jV0(u )) and P (1) = Vmax P (R(u )jV1 (u )). If P (0) > P (1) then
V0 (u )
n
n n
1( ) un
n n
the de
ision is u^ = 0, if P (0) < P (1) then the de
ision is u^ = 0, if P (0) = P (1) then the
n n
de
ision is arbitrary. In the analysis we will assume that the de
ision is erroneous in this
latter
ase, whi
h in
reases the bound on the error probability. The
onsidered algorithm
is an analog of maximum likelihood de
oding in the l0 {trun
ated {graph. Let us esti-
mate the probability of erroneous de
ision about symbol u . Without loss of generality we n
of the symbol u = 1 being inheritors of the family name and the
orresponding
ode
n
symbols. In this
onsideration we will take into a
ount only one dire
t des
endant (if it
exists) of a male symbol u 2 V~1(u ), namely, either the des
endant
losest to the symbol
m n
u from the left or the des
endant
losest to the symbol u from the right. Let w V1 (u )
m
~ m n
be the Hamming weight of the set V~1 (u ). The union bound gives the inequality
n
D (V~1 ( )) ;
X
Pb = P (^u = 1) < n
w
(18)
un
V~1 (u ) n
p
where D = 2 p(1 p) is the Bhatta
haryya parameter and the summation is taken over
all possible sets V~1 (u ). Let us show that
n
Pb < 2
1 a D2+ + a D ; I
I
(19)
1 a
where a = 12 2 .D
D
Assume I = l0 = 1. If family name extin
tion o
urs in the rst generation of the set
V~1(u ), all dire
t des
endants of u are zero symbols. There exist two
ombinations V~1 (u ),
n n n
both of weight 2+, whi
h
an
ause a de
oding error. An upper bound on the probability
of error,
aused by ea
h of the
ombinations is equal to D2+. If family name extin
tion
does not happen in the rst generation then an error is
aused by one of the
ombinations
of information symbols like fu = 0; : : : ; u 1 = 0; u = 1; u +1 = 0; : : : u + 1 =
n n n n n j
0; u + = 1; u + +1 = 0; : : : ; u + = 0g or fu = 0; : : : ; u 1 = 0; u = 1; u +1 =
n j n j n n n j n j n j
aused by these ombinations are upperbounded by D2+ . Summing D2+ over all j 1 j j
13
and adding an upper bound for the
ase of family name extin
tion we get for I = 1
Pb <
2D2 D + 2D2+ : (20)
1 D
In the general
ase we use analogous arguments. Consider all
odewords V~1 (u ), where
n
family name extin
tion o
urs in the lth generation, l 2 f1; 2; : : : ; l0g. The probability of
error,
aused by at least one su
h a
odeword, is upperbounded by
1
2 D2 l
2 1 D D2+ = 2a 1D2+ : l
(21)
The summation over all l; 1 l l0 = I; gives an upper bound on the probability of
error,
aused by at least one of the
odewords where family name extin
tion o
urs, whi
h
is equal to the rst term in (19), namely
2 11 aa D2+ :
I
(22)
The probability of error,
aused by at least one of the
odewords V~1 (u ), where family
n
the signal{to{noise ratio per bit. D < 1=2 is a suÆ
ient
ondition for the exponential
14
onvergen
e of the bit error probability to zero with the number of iterations I . Thus the
signal{to{noise ratio in the
hannel should ex
eed
Eb
N0 0
= 3 ln2 = 2:08 (3:18 dB) : (25)
The value (Eb=N0 )0 is an upper bound on the iterative limit (Eb =N0)0 for the signal{
to{noise ratio in the AWGN
hannel.
Results of the
al
ulation of the iterative de
oding bit error probability Pb as a fun
-
tion of the number of iterations I = l0 , using Monte{Carlo methods, are presented in
Figure 2. The
ase of transmission with memory one turbo{
odes over the AWGN
hannel
is
onsidered. It is easy to see that the theoreti
al bound on the iterative limit is in good
agreement with the numeri
al
al
ulations. Moreover, the slope of the
urves
orresponds
to the theoreti
al estimation:
d(log10 Pb ) 2 D2 1
dI
log10 1 D ; D< :
2 (26)
The lower bound on the minimum distan
e of memory one turbo{
odes obtained in
this paragraph will be strengthened and generalized to arbitrary turbo{
odes in the next
paragraph. Moreover, the
onstru
tive algorithm for the analysis of the asymptoti
prop-
erties of a permutor, studied in x 3, will be modied to a synthesis algorithm for \good"
permutors for turbo{
odes.
Codes
Consider a modi
ation of the method for permutor
onstru
tion, des
ribed in x 3, in order
to obtain turbo{
odes whose minimum distan
e is proportional to the logarithm of the
ode
length N .
Consider a
y
le = i1; i2 ; j2; j3 ; i3; i4 ; : : : ; j ; j1 and the sequen
e of distan
es d1 =
l
15
Denition 6 The summary distan
e dsum of the
y
le is
l
X
dsum = di : (27)
i =1
The summary distan
e of a half{
y
le
an be dened analogously.
Lemma 3 There exists a permutor matrix of size N N , su
h that the summary distan
e
of any of it's
y
les satises the inequality
dsum
log2 N 1 : (28)
log2 3
Proof: For any xed dsum, a matrix without
y
les with summary distan
e less than dsum
an be built using a modi
ation of the
onstru
tive pro
edure, des
ribed in x 3. The
dieren
e is that
y
les and half{
y
les with summary distan
e less than dsum will be
onsidered as prohibited.
Let us estimate nr(dsum), the maximal number of symbols in a row of the matrix
, that
an be marked by U (or 0U ). Consider an arbitrary row k. Let us
ount the
number of positions (k;
) in the row su
h that pla
ing a 1 in this position will
reate a
k
Æ = dL (i1 ; i2 )+ dL(j2 ; j3 )+ : : : + dL(j ; j1 ), the number of su
h positions does not ex
eed the
l
we
an l times
hoose between two dierent dire
tions to move. Analogous
al
ulations
an be done for half{
y
les. Then the total number of positions,
ausing prohibited
y
les
and half{
y
les with summary distan
e Æ, does not ex
eed
X Æ 1
Æ
l 1 2 = 2 3 1:
l Æ
(29)
l =1
We prohibit all
y
les and half{
y
les with summary distan
e Æ < dsum and therefore
dsum
X 1
nr (dsum) 2 3 1 = 3 sum
Æ d 1 1: (30)
Æ =1
An analogous bound
an be derived for n
(dsum), the maximal number of symbols in a
olumn of the matrix , that
an be marked by U (or 0U ).
16
For xed N
hoose some dsum, satisfying
2 3 sum 1 N < 2 3 sum
d d
: (31)
It follows from 2 max(n
(dsum); nr(dsum)) = 2 3 sum 1 2 < N (see Appendix B), that
d
the emergen
y pro
edure
an always be performed and it is possible to
onstru
t a matrix
, whi
h does not
ontain
y
les with summary distan
e less than dsum. From the right
hand side of (31) we get (28). The lemma is proved.
Consider now a turbo{
ode with permutor matrix , satisfying the
ondition of Lemma 3
with
omponent
odes whose a
tive
olumn distan
e satises the inequality
a
j
(j + 1); j 0 ; (32)
where
is some positive
onstant. This
ondition is satised for all re
ursive systemati
onvolutional en
oders.
Theorem 3 For any positive integer N there exists a turbo{
ode with information sequen
e
length N whose minimum distan
e satises
Proof: Assume the a
tive
olumn distan
e of the
omponent
odes satises inequality (32).
Consider an arbitrary nonzero information sequen
e u. For this sequen
e it is always
possible to nd a
y
le = i1 , i2 , j2 , j3, : : :, j , j1 , su
h that u = 1; k = 1; : : : ; l, and the
l ik
omponent en
oders do not visit the zero state within the intervals between the positions
i1 and i2 , j2 and j3 , : : :, j and j1 . The summary distan
e of this
y
le is lowerbounded
l
by inequality (28). Making use of (32) we nd that the weight of the
ode sequen
e v is
lowerbounded by
w (v )
2 log 3 (log2 N 1) ;
2
(34)
whi
h implies (33). The theorem is proved.
17
The existen
e of turbo{
odes whose minimum distan
e grows logarithmi
ally with the
ode length follows from (33). In
ombination with the paper [15℄, whi
h proves that the
minimum distan
e of turbo{
odes
annot grow faster than logarithmi
ally with the
ode
length, Theorem 3 determines the minimum distan
e behavior of optimal turbo{
odes with
respe
t to their
ode length.
In the proof of Lemma 3 a
onstru
tive method was used to analyze permutors, for
whi
h the summary distan
es of all
y
les satisfy (28). A modi
ation of this method
was used in order to
onstru
t turbo{
odes with lengths N = 500; 1000 and 2000 with
memory m = 2 re
ursive systemati
onvolutional
omponent en
oders. Simulation results
for transmission with these
odes and
odes with uniform interleavers of
orresponding
lengths over the AWGN
hannel are given in Figure 3. It is easy to see that the bit error
probability for turbo{
odes with spe
ially
hosen permutors is at least ten times smaller
than the bit error probability for turbo{
odes with uniform interleavers of
orresponding
length at a signal{to{noise ratio of 2 dB.
Let us assume, that Lemma 1 is wrong and the turbo{
ode
orresponding to the
y
le{
free permutor , appearing in the
ondition of the lemma, is degenerated in the lth gen-
eration, l l0 . This implies that there exists an (u 0 ; ){
lan that is degenerated in the
i
lth generation, i.e. some symbol u from the lth generation of the
lan appears one more
il
time either as the head of the
lan or as a des
endant in the kth generation, k l. We
will
he
k all possible
ongurations of the
lan and show that the matrix in all
ases
ontains at least one of the
y
les, listed in the
ondition of the lemma.
Case 1: The symbol u
oin
ides with it's an
estor in the kth generation, k < l,
il
(Figure 4 (a)). Let us denote the symbols along the path from u down to the symbol
ik
number of the en
oder) of the symbol u ; r = k; : : : ; l 1 (the head of the family), and
ir
18
generation, all the indi
es i ; i +1; : : : ; i 1 are dierent. It follows from the stru
ture of
k k l
the
ode that f 6= f +1 for any r. Consider now two sub
ases.
r r
2. This implies that the sequen
e i 1 ; i +1, j +1; j +2, i +2; : : : ; j 1 is a rst type {
y
le
l k k k k l
Case 2: There exists a symbol that is a des
endant of the
lan head in the lth generation
from one marriage and a des
endant of the
lan head in the kth generation from another
marriage, k l, and this symbol is not a des
endant of itself (Figure 4 (b)), i.e. u = u 0 . il ik
Let us denote by u 0 ; u 1 ; : : : ; u 1 ; u the symbols along the path from the
lan head u 0
i i il il i
(u 0 ; ){
lan is nondegenerated up to the (l 1)th generation. This
ase
an also be divided
i
f 1 = 2; f 0 1 = 1
an be
onsidered analogously.)
l k
19
Case 3: There exists a symbol whi
h is a des
endant of some symbol u from the in
marriage fe in the lth and kth generation of the
lan, 0 n < k l, su
h that it is not a
des
endant of itself and it does not
oin
ide with des
endants of the
lan head from another
marriage up to the lth generation. (Figure 4 (
)). Let f denote the type of the family of r
symbol u ; r = n + 1; : : : ; l 1 (the head of the family), f 0 denote the type of the family
ir q
All the indi
es i +1 ; i +2; : : : ; i 1,i0 +1; : : : ; i0 1; i0 are dierent sin
e the (u 0 ; ){
lan is
n n l n k k i
nondegenerated up to the (l 1)th generation. This
ase
an be divided into four sub
ases.
Case 3.1: k = n + 1; f 1 = 1; fe = 2. Then dL (i ; i 1 ) , dL (j 1 ; j 2 ) , : : :,
l l l l l
analogously.)
Case 3.2: k = n + 1; f 1 = 1; fe = 1. Then dL (i ; i 1 ) , dL (j 1 ; j 2 ) , : : :,
l l l l l
1. It follows from i = i0 +1 that dL(i +1 ; i0 +1) 2, dL(i 1 ; i ) ) dL(i +1; i 1) 3.
l n n n l l n l
This implies that the sequen
e i +1; i 1; j 1 ; : : : ; j +2 ; j +1 is either a rst or a se
ond type
n l l n n
analogously.)
Case 3.3: k > n + 1; f 1 = 1; f 0 1 = 2. Then dL (i ; i 1 ) , dL (j 1 ; j 2 ) ,
l k l l l l
2; f 0 1 = 1
an be
onsidered analogously.)
k
20
rst type {
y
le of length l + k 2n 2 2l0 2. (The
ase k > n +1; f 1 = 2; f 0 1 = 2 l k
an be
onsidered analogously.)
We
onsidered all possible
ases of degeneration of a
lan. All of them result in the
existen
e of either a rst type {
y
le of length less than 2l0 + 1 or a se
ond type {
y
le of length less than l0 . This implies that the
onstraints, imposed by Lemma 1 to
the permutation matrix , are suÆ
ient for (l0; ){nondegeneration of the
orresponding
turbo{
ode. Lemma 1 is proved.
reate a prohibited {
y
le and also prove that if inequality (3) is satised, we always
an
nd su
h an i0 that 0 = 0A, ( 0) = 0A.
i ;
k k; i
The rst statement we will prove by ontradi tion. Let us assume that setting 0 i ; k
both points, (i0;
) and (k;
0 ), as verti
es, sin
e the 0A's formerly in these positions
k i
indi
ated that no prohibited {
y
les existed through either point alone. Let this {
y
le
be = fi1 = k; i2; j2 ; j3 : : : ; i = i0 ; : : : ; i ; j ; j1 =
0 g.
n l l i
{half{
y
le of length n l with free vertex (k;
0 ) before the emergen
y pro
edure.
i
This {half{
y
le was prohibited. If it was a rst type {half{
y
le then it was
prohibited sin
e n l 2l0 . If it was a se
ond type {half{
y
le then was a se
ond
type {
y
le. This leads to the
onstraint n l l0 1, whi
h is satised only if was 0
a prohibited { y le. This ontradi ts the fa t that the free vertex of the {half{ y le 0
ited {
y
le before the emergen
y pro
edure. This is in
ontradi
tion with the way of
onstru
tion of the matrix.
We proved that the emergen
y pro
edure does not
reate prohibited {
y
les.
21
Let us show that if (3) is satised, it is always possible to perform the emergen
y
pro
edure. We denote by nr(l0 ; ) the maximal number of symbols in a row of the matrix
marked by U or 0U and by n
(l0 ; ) the maximal number of symbols in a
olumn of the
matrix marked by U or 0U . We will show that if N 2 max(nr(l0 ; ); n
(l0; )), it is
always possible to nd an i0 su
h that 0 = 0A, ( 0) = 0A.
i ;
k k; i
Consider a row of the matrix (say the kth) where we
annot nd symbols A. If N
2 max(nr(l0; ), n
(l0; )) then at least one{half of the symbols in the row are symbols
0A. Assume that some symbol at position (k; j ) is marked by 0A. This means that there
is a 1 at position ( 1(j ); j ) in the j th
olumn. If the symbol at position ( 1(j );
) is k
0A then we have found i0 = 1(j ). Assume that for all j su
h that = 0A the symbol
k;j
1 ( ) 6= 0A. Then the fra
tion of symbols 0A in the
th
olumn is less than one{half
j ;
k k
(the symbol at position (k;
) is U ). This fa
t is in
ontradi
tion with our assumption
k
N 2 max(nr (l0 ; ); n (l0 ; )), whi h implies that we an nd an i0 su h that 0 = 0A,i ; k
( 0 ) = 0A .
k; i
Let us now estimate nr(l0; ). Consider an arbitrary row k. We will
ount the maximal
number of positions (k;
) in this row su
h in whi
h a 1 would
reate a {
y
le =
k
...,dL (j ; j1 ) .
l
and
onsequently there are at most (2) positions (k;
) in the kth row whi
h
an be a
l
k
22
type {
y
le of length l, at most (2)( 1) (1 + l 1 + (l 2)(l 1)=2) positions whi
h
l
l =1 l =1
2l0(2)2l0 1 + l0 + l0(l02 1) + l0(l0 1)(2)2l0 8l03(2)2l0 : (35)
The same bound is valid for n
(l0 ; ). As it was shown, the emergen
y pro
edure
an
be performed if N 2 max(nr(l0 ; ); n
(l0; )). If we substitute the bound (35) in this
inequality we see that a permutor matrix without prohibited
y
les
an be
onstru
ted for
any N satisfying (3). The lemma is proved.
23
Referen
es
[1℄ Berrou C., Glavieux A., Thitimajshima P., Near Shannon Limit Error Corre
ting Cod-
ing and De
oding: Turbo{Codes. // Pro
eedings IEEE Int. Conf. on Communi
ations,
Geneva, Switzerland, 1993, p. 1064{1070.
[2℄ Gallager R. G., Low{Density Parity{Che
k Codes. M.I.T. Press, Cambridge, Massa-
husetts, 1963.
[3℄ Zyablov V. V., Pinsker M. S., Estimation of the Error{Corre
tion Complexity of
Gallager Low{Density Codes. // Probl. Pereda
h. Inform. 1975. V. 11, N. 1, P. 23{36.
[4℄ Margulis G. A., Expli
it Constru
tions of Graphs Without Short Cy
les and Low Den-
sity Codes. // Combinatori
a, 2(1):71{78, 1982.
[5℄ Margulis G. A., Some Expli
it Constru
tions of Low{Density
odes. // Pro
. III Inter-
national Seminar on Information Theory \Convolutional
odes;
ommuni
ation with
many users". So
hy 1987. P. 120{123.
[6℄ Margulis G. A., Expli
it Group{Theoreti
Constru
tions of Combinatorial S
hemes and
their Appli
ation to the Design of Expanders and Con
entrators. // Probl. Pereda
h.
Inform. 1988. V. 24. N. 1. P. 51{60.
[7℄ Tanner R. M., A Re
ursive Approa
h to Low Complexity Codes. // IEEE Trans.
Inform. Theory. 1981. V. 27, N. 9, P. 533{547.
[8℄ Benedetto S., Montorsi G.,Unveiling Turbo Codes: Some Results on Parallel Con-
atenated Coding S
hemes. // IEEE Trans. on Inform. Theory, 1996, V. IT{42, 2, P.
409{428.
[9℄ Lentmaier M., Truha
hev D. V., Zigangirov K. Sh., On the Theory of Low{Density
Convolutional Codes II. // submitted to Probl. Pereda
h. Inform.
[10℄ Engdahl K., Zigangirov K. Sh., On the Theory of Low{Density Convolutional Codes
I. // Probl. Pereda
h. Inform. 1999. V. 35. N. 4. P. 12{27.
24
[11℄ Stahl P., Anderson J., Johannesson R., A Note on Tailbiting and Systemati
Feedba
k
En
oders. // submitted to IEEE Trans. on Inform. Theory.
[12℄ Bahl L., Co
ke J.,Jelinek F., Raviv J.,
Optimal De
oding of Linear Codes for Mini-
mizing Symbol Error Rate. // IEEE Trans. Inform. Theory, 1974, V.20, 1, p. 284{287.
[13℄ Zigangirov K. Sh., Lentmaier M., Mathemati
al Analysis of an Iterative Algorithm for
Low{Density Code De
oding. // Probl. Pereda
h. Inform. 2000. V. 36. N. 4. P. 35{46.
[14℄ Johannesson R., Zigangirov K. Sh., Fundamentals of Convolutional Coding. Pis
at-
away, N.J., IEEE Press, 1999.
[15℄ Breiling M., A Logarithmi
Upper Bound on the Minimum Distan
e of Turbo Codes.
// submitted to IEEE Trans. on Inform. Theory .
25
Figure Captions
26
un
g repla ements 2
Figure 1:
27
−1
10
−2
10
−3
10
Pb
−4
10
−5
g repla
ements
10
−6
10
0 10 20 30 40 50 60 70 80 90 100
I
Figure 2:
28
−1
10
−2
10
−3
10
Pb
−4
10
−5
10
−6
10
g repla
ements
−7
10
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Eb =N0
Figure 3:
29
ui k
ui0 fe
fk f0
0
f0 ui0 +1
ui01 ui
ui +1 ui1 +1
n
f10 fn0 +1
n
k
fk+1 f1 fn+1
ui02 ui0 +2
ui +2 ui2 ... ui ... n
... k
... ... n +2
ui0 1
ui0 1
ui 1 ui 1
k
ui 1
k
fk0 1 fk0 1
l l
l
fl 1 fl 1 fl 1
ui0 ui0
ui = ui
l k ui = ui0
l k
k ui = ui0
l k
k
(a) (b) ( )
Figure 4:
30