The Viterbi Algorithm

It is known as a method in solving the convolutional code whereby it is first introduced and
later on this method had been improvised by the creator himself. On the other hand, the
Viterbi Algorithm (VA) can be termed as a dynamic programming algorithm. How a VA
works is basically very simple as it reduced the probability of error by making a comparison
on the possibility of occurrence for a set of transition state. The decision is made on which of
the occurrence has the highest probability. Alternatively, Stack Sequential Decoding
Algorithm (SS-DA) which needs less number of hardware to be implemented, serves as target
tracking purpose and high rate convolutional decoding. Both of these algorithms are
important not just in communication; however they bring about significant contributions on
the diverse areas. These diverse areas are namely, handwritten word recognition and through
to non-linear estimation.
The algorithm
A trellis which often represented by a Finite State Diagram (FSM) is used meant for a VA to
find its shortest path by providing the necessary set of observations. The nodes represent the
states and the edges represent possible transitions between two states.
Trellis diagram spread over time

Corresponding state diagram over time

(FSM)

A corresponding outcome is generated for each of the possible transitions by the FSM which
viewed as symbols observation containing some of the original corrupted data symbols by
noise and it is not necessarily to be a binary digit, it can also be either a letter or an alphabet.

However, it is essential for the noise to exist permanently in the observation as it is

transmitted along the channel.
Hidden Markov Model (HMM)
The FSM is not noticeable as it is hidden away from the VA. The only way to view it is by
observing the outcome produced by the HMM. The application of such model can easily
track target and recognize characters where estimation of possible true states of the system is
done. All the estimations of HMMs underlying are done by the trellis in HMM and
transitions. The similarity of both MM and HMM models is that both models uses a set of
metrics mainly to help the VA to make decision on which path that is most likely to have been
followed and costing various path through the trellis

and it definitely links with the

observation and transitions within the FSM.

Symbols of VA
Symbols
t
N
xn
ot

Definition
Discrete time index
Total number of states FSM
Nth state of FSM
Observation symbol at time, t (M number of

spnt

symbols
The survivor path which terminates at time t,
in the nth state of the
FSM.(consists of an ordered list of xn's
visited by this path from time t = 0

T
n
anm

to time t.)
Truncation length of VA
Initial state metric for the nth state at t = 0
Probability that given that state xm
occurs at time t - 1, the state x n will occur at

bn

time t
Probability that the observation symbol ot
would occur at time t, given that we are in

nt

the state xn at time t

Product of the metrics (n, anm and bn) for
each transition in the nth survivor path, from
time t = 0 to time t

Conversion of Viterbi algorithm into Pseudocode

Initialization
t = 0;

Calculation.
For all t, where 1 t T,

Decision
If t = T,

For all n, where 1 n N

For all n, where 1 n N

For all n, where 1 n N

n0 = ln n;

For all m, where 1 m N

T = Max [nt];

spn0 = [xn];

nt = Max [mt-1 + ln anm + End For;

End For;

ln bn];

spT = spnt

End For;
spnt = Append[xn,spmt]

For,
nt = T;

For,
mt-1 + ln anm + ln bn = nt;
End For;
End For;

Say, in a scenario where there are two or more transitions which are found to be maximum
whereby their metrics are equal, then both the transitions is chosen at random as the most
likely transition and will be assigned to the state survivor path metric. On the other hand,
what is left will be te leftover transitions where it will eventually be discarded where it is
later on append onto another survivor path of state t-1 where the transition originated from.
As the state is being incremented, the state t-1 then become state of time and is to be
examined. The same process take place even when the VA moves to a transition of t+1, as the
operations are the same regardless of which transitions it appears to be as all operations are
only carried out at time t. As previous, the VA will only determines and decides on which
most likely survivor path is to be taken after the it reaches the truncation length, t=T with the
purpose of finding the survivor with the greatest metrics. The whole process repeats again if
there is found more than one survivor has the greatest metrics again. This survivor path be
indicate as the outputs, spT, by the VA along with its survivor metric, GT.
Examples
Communication Channel Set Up

The diagram shows a convolutional code decoding from a memoryless Binary Symmetric Channel
(BSC). The input system denoted in I, in sequence of binary digits are to be transmitted along the
communication channel. A shift register in the encoder shifts a number of bits from I once at a time
to create a set of outputs which undergoes logical operation done on parts of I in the register
memory known as the convolutional encoding process. Redundancy is introduced resulting output
bits. Bits by bits are shifted along the register as transmitted output symbols and are observed by
the VA which contained the memory of the previous bits in I to detect any corruption that occurred
to enable the VA to decode the original bits from the previous and subsequent observation symbol.
Convolutional Encoder

The convolutional encoder by the boxes above can easily be represented by FSM. The shift
register holding the contents can are represented by the boxes which is the current state of the
FSM. This state is when the FSM first hold the actual content of the shift register.

Encoder can be represented by the FSM

For an example in a scenario where the current state is 01, both S1 and S2 will filled up with bits 1
and 0 respectively. Transitions between states are denoted with the lines with arrows labelled as x/y
with x to be the two digit binary number as the output symbol to be sent across a communication
channel for transitioning purposes. On the other hand, y denotes the binary digit from I, causing
transitioning to take place when sent across a communication channel and was then shifted into an
encoder. However, there will be a scenario where if we transmit bit 0 it is possible that it could be
interpreted by the receiver as a 1, and vice versa due to the corruption of noise that inherent to the
channel during encoder transmission. In order to form the actual data sequence sent, noise is used
along with a trellis diagram of the known FSM.
Trellis diagram for FSM spread over time

From the example shown previously, VA can use trellis to display nodes used with fixed time
progresses. As mentioned, grey line is used for bit 1 and the black is used as bit 0. These lines
produce a two bit binary digits as labelled next to the transitions that produced them and normally
these transitions and states are fixed in the range of the differing time intervals.

Recursive structure of this trellis diagram

As proven by Viterbi, it is possible to have

Hamming Distance (number of bits to reduce the
survivor Metrics as determined with the log
likelihood function to a minimum distance
measure. Hamming Distance can best defined as
the number of bits that are different between the
symbol as observed by VA and the output symbol
of a dependent convolutional encoder on input
sequence. In short this measurement can be
derived as combined measurement for both anm
and bn in each transition found in the trellis. As
decoding take place, the n's are set as 0 indicating
it is in the normal start state while the other states
with a n with a value close to Few assumptions
have been made in this example where, the
encoder state will always be in state 00 with 0 = 0
and all n's are set to 100.
States Table (Table 1)

The outcome of 00 11 01 01 11 00 as obtained from the encoder with an input sequence of I (0 1 1

0 0 0) being transmitted across a BSC and these outcomes are named as the Encoder Output
Sequence (EOS). The table clearly shows all the content in correspond to each memory in the shift
register with certain assumptions on the initialization of a zero values as the beginning of the
encoding process. Transmission of EOS either for construction purposes or to be send across the
channel are done simultaneously. Viterbi Algorithm would find the most possible set of states that
the actual FSM tends to visit to determine the actual input sequence from the set of output received
that eventually repeated itself (01).

The Viterbi Algorithm

Transitions and Metrics at t = 1

Survivor Paths and Survivor Path Metrics at t

=1

Transitions and Metrics t = 2

Survivor Paths and Survivor Path Metrics at t

=2

Transitions and Metrics at t = 6

Survivor Paths and Survivor Path Metrics at t = 6

Using alphabetical representation with a = 00, b = 01, c = 10, d = 11, to represent 4 states and
initially we set t as1. n's are assigned in corresponding to n0, where a0 = 0 and b0 =
c0 = d0 = 100 with all their transitions solved. Say, in a scenario where VA observed a
symbol of 01 at t = 1, it is determined that the transition distance for the hamming from a at t
= 0 to t = 1.The total metrics will be determined once the transition distance is found. Let us
visualise this in the diagram below, with a at t =1 forms 2 possible transitions. First transition
will be from c which generates a metric of 101 and another from a with a metric of 1
regardless of which states at time t = 1. Best transition will be selected into each states
provided that it is a transition from a at t = 0 for its metric is the minimum among both the
transitions converges to a when t 1. spn1 and n1 for each state were set by VA introducing
a condition of spa1 = {a,a} and a1 = 1. From the figure, it shows the survivor path lengths
being selected when t = 1. The whole process eventually recurs itself for every time steps
with their decoding and survivor stages are shown as in the figure at t = 2, 6. A decision has

to be made for the most likely path once VA reached time T = t with the smallest Hamming
Distance. Typically in this example the value of 6 is used as the T as assumption in order to
result the eliminating path at state a to obtain a new minimum Hamming distance of 2,
making it the most possible path to be taken. The estimated sequence of binary digits in
sequence of 0 1 1 0 0 0 in correspond to the actual input sequence will be outputted as though
they are part of the actual input sequence. In this case, the input sequence has been recovered
regardless of the error introduced throughout the transmission.

Algorithm extension
VA is dependent on the input of the demodulator which mostly carries out the task of making
the tough selection of the symbols being received from the channeling medium. This implies
to regardless of which modulation it is, let it be a phase, frequency or even an amplitude
modulation. However in the midst of making a hard decision, sometimes the demodulator
does make soft decision as an extension where soft outputs are produced. You may need to
imagine it in a case where a demodulator instead of producing either bits 0 or 1, it is made up
of symbols which basically made up the confidence of the transmitted symbols and literally
increases the information received in VA for performance boost. Similar to hard decision, it is
the VA role to decode all these soft decision and later on produces the soft output decision
known as the Soft Output Viterbi Algorithm (SOVA) based on the Viterbi algorithm.
Normally in this version of VA, the decision is made separately where the most likely
survivor path was decided by hard decision while at the same time the information of
confidence of each symbols tends to occur along the path is generated. Bouloutas et al [12].
Bouloutas's will be another extension where manage to overcome the error during the
insertions and deletions based on the set outcome as well as the changes in symbol as
detected by the VA. This method implies the FSM where first produced the symbols then a
product trellis that compensates for the insertions and deletions in the observation sequence.
Metrics are assigned for all the insertion, deletion and the changes in the assigned operation
metrics whereby it is dependent of the applied VA. The most likely path is produced by the
VA through the FSM states which estimates of the actual data sequence and also the best
sequence of operations performed on the data to troubleshoot the incorrect observation
sequence. In fact, practically there will be a lot of errors being produced during code
implementation where sometimes, the programmer tends to miss out certain character and
this is when the VA become handy as the correction programming code whilst compiling

programs. The last extension of VA will be the parallel version as introduced by Fettweis and
Meyr, and Lin et al meant for fast hardware implementations of VA.

The applications
Communications
VA is introduced mainly to overcome occurrence of Intersymbol Interference (ISI) in
modulation system causing the demodulator to miss out and wrongly interpret certain
symbols that tend to run into each other or consecutive symbols that disperse. Estimation of
possible input sequence of 0s or 1s in modulator at the transmitter end of the channel where
the observation symbols often affects the sequence of IS. In this case, VA is passed onto the
next VA for the actual sequence of the input from the transmitter and this scenario is normally
known as the Viterbi Equalizer whereby its usage is in conjunction with Trellis-Coded
Modulation (TCM) depending on the types of channel being used. SOVA in this case can
utilized this advantages to transmit more information into the decoder. TCM is first introduce
by Ungerbocck whereby convolutional coding which add in redundancy is no longer
acceptable due to limited bandwidth that reduce data transmission. TCM can improvise this
by implementing both auto-correction capability with addition of redundancy into the
modulation signal using few of the amplitude levels. Bits are mapped onto various amplitudes
or phases in the modulation to keep the bandwidth signal unchanged. Similarly, the most
likely set of bits in convolutional codes will be produced by decoding TCM codes with VA.
However, those decisions being made are just soft decisions which are used in VA for
decoding purposes. In conclusion, VA plays an important role as an equalizer in the
application of communication in seeking for a better TCM performance in various types of
environment.
Target tracking

Above diagram shows the tracking system as proposed by Demirbas. Estimation of the
targets state (speed, position, acceleration) produced by VA and this method constructs a
trellis diagram with an FSM with both next states and states transitions using a typical motion
model which functions as a non-linear motion estimator. With such model, it has become far
more convenient to produce the next set of possible states within the transitions of these
states. The n-dimensional vector is represented by each and every states of the trellis and
these dimensions are mainly a specific range, bearing or even an elevation position of a
plane. This differ from a Kalman filer where it needs only a non-linear motion model. The
states number can vary as it proceed to the next corresponding state whereby is shows
flexibility as compared to previous version of VA with fixed amount of states throughout the
trellis. It is known that the VA can only consider a certain limited amount of possible states to
move in although in theory we have an infinite amount of possible states. Trellis diagram is
constructed with both anm and n metrics known to track down the interference or a normal
background noise. The next step came in when the VA predict the next possible path take by
this trellis through observation sequence from the sensor. Later on, this method was further
improvised by Demirbas termed as the Stack Sequential Decoding Algorithm for speedy
estimation. Besides of being the sub-optimum to a VA user, it can be modelled to implies on
any no-linear dynamic system (populations, growth, economics) just that it is used on
manoeuvrable targets through radar system by Dermibas. This system is applicable onto
different dimensions with their individual trellis diagram consisting of different point of
estimation. These components were done separately depending on the dimensions at that
time.

It is even possible to be figuring out the missing observation in an instance when a plane goes
out of a radar range momentarily. With these observations, it is eas for the VA to detect the
actual position. However, it is also crucial to know a series of number of previous observation
in support of obtaining a trustable set of information on the exact position similar to ISI. The
tracking method was advance to an extend where it can now measure frequency of a signal by
th Fast Fourier Transform of signal received where the states in HMM represents a certain
frequency where it could track down the changes in frequency level used by some radar for
location detection purposes which made it differ from Demirbas method. This advancement is
yet to come to an end where it cannot still carry out the task of multiple tracking but was
eventually improvise by Demirbas and Streit. Kalman filter is needed to estimate the
parameters (normally represented by random variable) needed in reality to be imply on the
manoeuvrering model as suggested by Demirbas. A multitrack VA was invented to track
down two crossing targets of a time-varying frequency so that altering the parameters of the
model to fit in this type of tracking could be done simpler.
Recognition
As can be seen so far for an instance, post code and address recognition of printed and
handwritten words are all simply the applications of VA. More of these examples that can be
observed are document analysis, car license plate recognition and direct input into a computer
with pen. One addition advantage of a model like this is to auto detect if there is a word
unfound in the system dictionary of a VA, then a less like sequence of letters will be provided
to ensure a higher semantic model is determining the word produced. Similar method is being
applied in OCR to determine the sequence if letters of printed character and for that VA is
required to deal with the recognition of individual characters in formation of a word. With
that, it deals with Chinese characters and the same recognition method is also applied on
English characters. Lower level of processing such as character segmentation in recognizing
which part of a paper is the background, which area contains a part of a letter and various
pattern recognition. These patterns are determination of edges and region segmentation of an
image. Last but not lease in neural networks where continuous speech could be recognize
with VA.