Theory of Computation

UNIT- I
INTRODUCTION
Symbol
A character, glyph, mark.
An abstract entity that has no meaning by itself, often called uninterrupted.
Letters from various alphabets, digits and special characters are the most commonly
used symbols.

Alphabet
A finite set of symbols.
An alphabet is often denoted by sigma, yet can be given any name.
B = {0, 1} Says B is an alphabet of two symbols, 0 and 1.
C = {a, b, c} Says C is an alphabet of three symbols, a, b and c.

String also called a Word

A finite sequence of symbols from an alphabet.
01110 and 111 are strings from the alphabet B above.
aaabccc and b are strings from the alphabet C above.
A null string is a string with no symbols, usually denoted by epsilon.

Formal Language, also called a Language

A set of strings from an alphabet.
The set may be empty, finite or infinite.

L(M) is the notation for a language defined by a machine M.

The machine M accepts a certain set of strings, thus a language.

L(G) is the notation for a language defined by a grammar G.

The grammar G recognizes a certain set of strings, thus a language.

M(L) is the notation for a machine that accepts a language.

The language L is a certain set of strings.

G(L) is the notation for a grammar that recognizes a language.

The language L is a certain set of strings.

FINITE STATE MACHINES

There are 3 kinds of computing machines, finite-state automata (FSA), pushdown
automata (PDA), and Turing Machines (TMs), and the languages that they can compute
are mutually inclusive:
A finite state machine (FSM, also known as a deterministic finite automaton or DFA) is
a way of representing a language (meaning a set of strings; we're interested in
representing the set strings matching some pattern).

FINITE AUTOMATA:
M = (Q, Sigma, delta, q0, F) is a definition of a Finite Automata.
To be fully defined the following must be provided:

Q a finite set of states often called q0, q1, ... , qn or s0, s1, ... , sn.

There is no requirement, in general, that every state be reachable.

Sigma a finite input alphabet

What is a DFA?

A deterministic finite automaton (DFA) is given by:

1. A finite set of states ,

2. A finite set of input symbols ,

3. A transition function ,

4. An initial state ,

5. A set of accepting states .

As an example consider the following automaton

where

The DFA may be more conveniently represented by a transition table:

The table represents the function , i.e. to find the value of we have to look at the
row labelled and the column labelled . The initial state is marked by an and all
final states are marked by .

Yet another, optically more inspiring, alternative are transition diagrams:

There is an arrow into the initial state and all final states are marked by double rings. If

then there is an arrow from state to which is labelled .

We write for the set of words (i.e. sequences) over the alphabet . This includes the
empty word which is written . I.e.

* of DFA and its Properties :

For a state q and string w, *( q , w ) is the state the DFA goes into when it
reads the string w starting at the state q. In general a DFA goes through a number of
states from the state q responding to the symbols in the string w. Thus for a DFA < Q ,
, q0 , , A > , the function

*:Q * -> Q
is defined recursively as follows:

Definition of *:
Basis Clause: For any state q of Q , *( q , ) = q , where denotes the empty string.

Inductive Clause: For any state q of Q, any string y * and any symbol a ,
*( q , ya ) = ( *( q , y ) , a ) .

In the definition, the Basis Clause says that a DFA stays in state q when it reads an empty
string at state q and the Inductive Clause says that the state DFA reaches after reading
string ya starting at state q is the state it reaches by reading symbol a after reading string
y from state q.

Properties of * :
We can see the following two properties of * .
Theorem 1: For any state q of Q and any symbol a of for a DFA < Q , , q0 , ,A
>,

*( q , a ) = (q,a)

The next theorem states that the state reached from any state, say q , by reading a
string, say w , is the same as the state reached by first reading a prefix of w, call it x, and
then by reading the rest of the w, call it y.

Theorem 2: For any state q of Q and any strings x and y over for a DFA < Q , ,
q0, ,A > ,
*( q , xy ) = *( *( q , x ) , y ) .

Recursive definition of *:

Basis Clause: *.

Inductive Clause: If x * and a , then xa *.

Extremal Clause: Nothing is in * unless it is obtained from the above two clauses.

DFAs in programming languages

When you are learning about models of computation, one simple model is a deterministic
finite automata or DFA for short.

Formally, the definition of a DFA is:

Q, a set of states
S, an single state which is an element of Q. This is the start state.
F, a set of states designated as the final states
Sigma, the input alphabet
delta, a transition function that maps a state and a letter from the input alphabet, to a
state

• DFAs are used to recognize a language, L. A language is a set of strings made

from characters in the input alphabet. If a language can be recognized by a DFA,
it is said to have a regular grammar.
• To use a DFA, you start in an initial state, and process the input string a character
at a time. For example, if the input alphabet consists of "a" and "b", then a typical
question is to ask whether the string "aaab" is accepted by a DFA.
• To find out whether it is accepted, you start off in the state state, S. Then you
process each character (first "a", then "a", then "a", and finally "b"). This may
cause you to move from one state to another. After the last character is processed,
if you are in a final state, then the string is in the language. Otherwise, it's not in
the language.
• There are some languages that can't be recognized by a DFA (for example,
palindromes). Thus, a DFA, while reasonably powerful, there are other
(mathematical) machines that are more powerful.

Often, tokens in programming languages can be described using a regular grammar.

FSM with output in hardware :
A finite state machine with output is similar to describe formally.

• Q, a set of states
• S, an single state which is an element of Q. This is the start state.
• Sigma, the input alphabet
• Pi, the output alphabet
• delta, a transition function that maps a state and a letter from the input alphabet, to
a state and a letter from the output alphabet.

The primary difference is that there is no set of final states, and that the transition
function not only puts you in a new state, but also generates an output symbol.

FSM with Outputs: Moore machines

. Moore machines add outputs to each state. Thus, each state is associated with an output.
When you transition into the state, the output corresponding to the state is produced. The
information in the state is typically written as 01/1. 01 indicates the state, while 1
indicates the output. 01/1 is short hand for q1q0 = 01/z = 1

The number of bits in the output is arbitrary, and depends on whatever your application
needs. Thus, the number of bits may be less than, equal, or greater than the number of
bits used to represent the state.

FSM with Outputs: Mealy machines

A Moore machine has outputs that are a function of state. That is, z = f( qk-1,..., q0 ).

A Mealy machine has outputs that are a function of state and input, that is That is, z = f(
qk-1,..., q0, xm-1,..., x0 ).

We usually indicate that the output is dependent on current state and input by drawing the
output on the edge. In the example below, look at the edge from state 00 to state 01. This
edge has the value 1/1. This means, that if you are in state 00, and you see an input of 1,
then you output a 1, and transition to state 01.

Thus, 1/1 is short hand for x = 1 / z = 1.

APPLICATION OF FINITE AUTOMATA

• Many other systems operating in practice can also be modeled by finite automata
such as control circuits of computers, computer network communication
protocols, and lexical analyzers for compilers etc.
• Many of those systems fall into the class of systems called reactive system.
• A reactive system is a system that changes its actions, outputs and
conditions/status in response to stimuli from within or outside it. It is an event
 driven or control driven system continuously having to react to external and/or
internal stimuli.
• Finite automata are formal and rigorous and computer programs can be easily
written to simulate their behaviors.
 COMMUNICATION:
The next example is a protocol for a computer to follow in communicating with
another computer. Again it is a very simplified version.

The language of a DFA

To each DFA we associate a language . To see whether a word

we put a marker in the initial state and when reading a symbol forward the marker along
the edge marked with this symbol. When we are in an accepting state at the end of the

word then , otherwise .

As an example we calculate :

Using we may now define formally:

 Hence we have that because and .

Language Accepted by DFA

*
A string w is accepted by a DFA < Q , , q0 , , A > , if and only if ( q0 , w )
A . That is a string is accepted by a DFA if and only if the DFA starting at the initial state
ends in an accepting state after reading the string.
A language L is accepted by a DFA < Q , , q0 , , A > , if and only if L = { w |
*
( q0 , w ) A } . That is, the language accepted by a DFA is the set of strings accepted
by the DFA.

Example 1 :

This DFA accepts { } because it can go from the initial state to the accepting state (also
the initial state) without reading any symbol of the alphabet i.e. by reading an empty
string . It accepts nothing else because any non-empty symbol would take it to state 1,
which is not an accepting state, and it stays there.

Example 2 : DFA with one cycle

This DFA has a cycle: 1 - 2 - 1 and it can go through this cycle any number of times by
reading substring ab repeatedly.
To find the language it accepts, first from the initial state go to state 1 by reading one a.
Then from state 1 go through the cycle 1 - 2 - 1 any number of times by reading substring
ab any number of times to come back to state 1. This is represented by (ab)*. Then from
state 1 go to state 2 and then to state 3 by reading aa. Thus a string that is accepted by this
DFA can be represented by a(ab)*aa .

Example 3 : DFA with two independent cycles

This DFA has two independent cycles: 0 - 1 - 0 and 0 - 2 - 0 and it can move through
these cycles any number of times in any order to reach the accepting state from the initial
state such as 0 - 1 - 0 - 2 - 0 - 2 - 0. Thus a string that is accepted by this DFA can be
represented by ( ab + bb )*.

Example 4 :
This DFA has two accepting states: 0 and 1. Thus the language that is accepted by this
DFA is the union of the language accepted at state 0 and the one accepted at state 1. The
language accepted at state 0 is b* . To find the language accepted at state 1, first at state 0
read any number of b's. Then go to state 1 by reading one a. At this point (b*a) will have
been read. At state 1 go through the cycle 1 - 2 - 1 any number of times by reading
substring ba repeatedly. Thus the language accepted at state 1 is b*a(ba)* .

There is a systematic way of finding the language accepted by a DFA and we are going to
learn it later. So we are not going to go any further on this problem here.

What is an NFA?
Nondeterministic finite automata (NFA) have transition functions which may assign
several or no states to a given state and an input symbol

belongs to its language ( ). Indeed we shall see that every NFA can be
translated into an DFA which accepts the same language.

Here is an example of an NFA which accepts all words over s.t. the symbol
before the last is .

A nondeterministic finite automaton (NFA) is given by:

1. A finite set of states ,

2. A finite set of input symbols ,

3. A transition function ,

4. A set of initial state ,

 5. A set of accepting states .

Hence, the only difference to DFAs is to have start states instead of a single one and the
type of the transition function. As an example we have that

where so given by

Note that we diverge he slightly from the definition in the book, which uses a single
initial state instead of a set of initial states. Doing so means that we can avoid introducing
-NFAs

The language accepted by an NFA :

consider the word (which is not accepted by ).

Initially we have

After reading we have to use two markers because there are two arrows from which
are labeled :

Now after reading 0 the automaton has still got two markers, one of them in an accepting
state:
However, after reading the 2nd 0 the second marker disappears because there is no edge
leaving and we have:

Which is not accepting because no marker is in the accepting state.

To specify the extended transition function for NFAs we use an generalization of the

union operation on sets. We define to be the union of a (finite) set of sets:

In the special cases of the empty sets of sets and a one element set of sets we define:

As an example

Actually, we may define by comprehension, which also extends the operation to

infinite sets of sets (although we don't need this here)

We define with the intention that is the set of states

which is marked after having read starting with the initial markers given by .

(3)

(4)

NFA with epsilon moves

Definition and example of a NFA with epsilon transitions.
Remember, epsilon is the zero length string, so it can be any where in the input string,
front, back, between any symbols.
 There is a conversion algorithm from a NFA with epsilon transitions to a NFA without
epsilon transitions.

Consider the NFA-epsilon move machine M = { Q, sigma, delta, q0, F}

Q = { q0, q1, q2 }
sigma = { a, b, c } and epsilon moves
q0 = q0
F = { q2 }
sigma plus epsilon

delta | a | b | c |epsilon
------+------+------+------+-------
q0 | {q0} | phi | phi | {q1}
------+------+------+------+-------
q1 | phi | {q2} | phi | {q2}
------+------+------+------+-------
q2 | phi | phi | {q2} | phi
------+------+------+------+-------

The language accepted by the above NFA with epsilon moves is the set of strings over
{a,b,c} including the null string and all strings with any number of a's followed by any
number of b's
followed by any number of c's. ("any number" includes zero)

Now convert the NFA with epsilon moves to a NFA

M = ( Q', sigma, delta', q0', F')
First determine the states of the new machine, Q' = the epsilon closure of the states in
the NFA with epsilon moves. There will be the same number of states but the names can
be constructed by writing the state name as the set of states in the epsilon closure. The
epsilon closure is the initial state and all states that can be reached by one or more
epsilon moves.

Thus q0 in the NFA-epsilon becomes {q0,q1,q2} because the machine can move from
q0 to q1 by an epsilon move, then check q1 and find that it can move from q1 to q2 by
an epsilon move.

q1 in the NFA-epsilon becomes {q1,q2} because the machine can move from q1 to q2
by an epsilon move.

q2 in the NFA-epsilon becomes {q2} just to keep the notation the same. q2 can go
nowhere except q2, that is what phi means, on an epsilon move.

We do not show the epsilon transition of a state to itself here, but, beware, we will take
into account the state to itself epsilon transition when converting NFA's to regular
expressions.
 The initial state of our new machine is {q0,q1,q2} the epsilon closure of q0

The final state(s) of our new machine is the new state(s) that contain a state symbol that
was a final state in the original machine.

The new machine accepts the same language as the old machine, thus same sigma.

So far we have for out new NFA

Q' = { {q0,q1,q2}, {q1,q2}, {q2} } or renamed { qx, qy, qz }
sigma = { a, b, c }
F' = { {q0,q1,q2}, {q1,q2}, {q2} } or renamed { qx, qy, qz }
q0 = {q0,q1,q2} or renamed qx

Conversion of NFA to DFA

Conversion of NFA to DFA

Let M2 = < Q2 , , q2,0 , 2 , A2 > be an NFA that recognizes a language L. Then the
DFA M = < Q, , q0 , , A > that satisfies the following conditions recognizes L:

Q = 2Q2 , that is the set of all subsets of Q2 ,

q0 = { q2,0 } ,

( q, a ) = for each state q in Q and each symbol a in and

A={q Q|q A2 }

To obtain a DFA M = < Q, , q0 , , A > which accepts the same language as the given
NFA M2 = < Q2 , , q2,0 , 2 , A2 > does, you may proceed as follows:

Initially Q = .
First put { q2,0 } into Q. { q2,0 } is the initial state of the DFA M.
Then for each state q in Q do the following:

add the set as a state to Q if it is not already in Q for each symbol a in .

For this new state, add ( q, a ) = to .

When no more new states can be added to Q, the process terminates. All the states of Q
that contain accepting states of M2 are accepting states of M.

Note: The states that are not reached from the initial state are not included in Q obtained
by this procedure. Thus the set of states Q thus obtained is not necessarily equal to 2Q2 .

Example 1: Let us convert the following NFA to DFA.

Initially Q is empty. Then since the initial state of the DFA is {0} , {0} is added to Q.
Since 2( 0 , a ) = { 1 , 2 } , { 1 , 2 } is added to Q and ( { 0 } , a ) = { 1 , 2 } .

Since 2( 0 , b ) = , is added to Q and ({0},b)= .

At this point Q = { {0} , { 1 , 2 }, } .

Then since { 1 , 2 } is now in Q, the transitions from { 1 , 2 } on symbols a and b are

computed. Since 2( 1 , a ) = { 1 , 2 } , and 2( 2 , a ) = , ( { 1,2 }, a ) = { 1,2 } .

Similarly ({1,2} , b ) = { 1 , 3 } . Thus { 1 , 3 } is added to Q .

Similarly ( { 1 , 3 } , a ) = { 1 , 2 } and ( { 1 , 3 } , b ) = . Thus no new states are

added to Q . Since the transitions from all states of Q have been computed and no more
states are added to Q, the conversion process stops here.

Note that there are no states of Q2 in . Hence there are no states that M2 can go to from

. Hence ( ,a)= ( ,b)= .

For the accepting states of M, since states 0 and 1 are the accepting states of the NFA, all
the states of Q that contain 0 and/or 1 are accepting states. Hence { 0 }, { 1 , 2 } and { 1 ,
3 } are the accepting states of M.

The DFA thus obtained is shown below.

Example 2: Similarly the NFA

is converted to the following DFA:

Sample DFA and NFA Questions
1. Let L1 be the set of strings over {a,b}* such that the number of a's is divisible by 2
and the number of b's is divisible by 3. For example, the following strings are in
L1:
aabbb
bbbababb
aa
aaabaaabaab
Construct an DFA M that accepts L1. Describe what each state in M represents.
Answer

• State A is the accepting state. It represents strings with an even number of a's and
a number of b's divisible by 3 (#b's = 0 mod).
• State B represents strings with an even number of a's and a number of b's = 1 mod
3.
• State C represents strings with an even number of a's and a number of b's = 2 mod
3.
• State D represents strings with an odd number of a's and a number of b's divisible
by 3 (#b's = 0 mod).
• State F represents strings with an even number of a's and a number of b's = 1 mod
3.
• State G represents strings with an even number of a's and a number of b's = 1 mod
3.

The state names indicate the meaning of each state. State Q10 is the state we are in after
seeing one 0 and zero 1's. State Q12 is the state we are in after seeing one 0 and two 1's.
State Qn0 is the state we are in after seeing more than two 0's and zero 1's.

We would need 25 states to recognize a language consisting of strings with exactly 3 0's
or 3 1's.
 2. Convert the following NFA into the equivalent DFA.

What language does this NFA accept? (Hint, The machine consisting of states
0,1,2 and 3 should look familiar).

Answer

The top half of the NFA is the x = 0 mod 3 machine from the homework. The bottom
half of the NFA is a x = 0 mod 4 machine. The NFA is the union of these two machines.
Therefore L(M)= binary strings x such that x = mod 3 or x = 0 mod 4.
3.(a) Convert the following NFA with /\ transitions into an NFA without /\
transitions.

(b) Convert the NFA into a DFA.

Answer

(a)
(b)