“Necessity is the mother of Invention”

Man created machines to reduce his laborious and

monotonous work and perform it with unparalleled
accuracy. However, until recently machines
couldn‟t do work which required thinking and
analytical skills.
This scenario has begun on the path of change.
History shows that the greatest technologies have
come as a result of inspiration from biology; ANN
(artificial neural network) is one such technology.
A Biological Neuron
Let‟s dive into the world of neural networks.
Example:
Man has always believed that, the human brain is Y = F (5A + 2B)
the greatest machine ever. It consists of a dense In the above equation, A has more „importance‟ in
network of tiny processing elements called deciding the output Y compared to B.
neurons; this network is the force behind the
amazing processing power that the brain has. Let‟s implicate the neuron mathematically,
Wouldn‟t it be wonderful, if machines also have
such power? Synapses will get modeled as multiplying factors
If we are able to implicate a neural network or weights; dendrites as input terminals; soma as
artificially, then, it would also be able to process the processing block; axon as the output terminal.
and analyze like the human brain. To create such a
network, we need to have artificial neurons. For
that, let‟s first have a look at the biological neuron.

The biological neuron has dendrites along which it

receives the inputs. These signals are then fed to
the soma (or nucleus) which processes them. In
effect, it sums the signals and transforms the
resultant by a function. The output is then sent
down the axon. The output signal is then given to
other neurons through links known as synapses.
The strength of the synapses decides how much
the signal gets weakened before reaching the Mathematical Model
next neuron; thus determining, the degree of
„importance‟ this signal has, in deciding the Now, a system can be represented in a block
output of the next neuron. diagram form like this:

An n-input m-output system
As the diagram shows, every desirable system
would have a relationship or formula between its
input and output.
If we connect multiple neurons with each other,
they would be able to establish such a relationship.
What if, the formula for a particular system
consists of functions other than the function
provided in the neurons
i.e. G ( ) is not same as F ( )?
Say the function provided in the neuron is addition;
now, multiplication can be performed using
addition, subtraction can be performed using
addition; integration is finally addition, and so
on…
Thus, by having appropriate multiplying factors or
weights, we can achieve any kind of a relationship
between the input and the output. However, care
should be taken to choose the function which
would suit the desired application i.e. F ( ) should
suit G ( ), so as to avoid the unnecessary usage of
more neurons.
Neurons are interconnected in layers. Let‟s create
one such network.
Let‟s have an input layer for buffering the inputs;
core (or hidden) and output layers, to give it the
ability to establish the complex relationships.
3 layers of Neurons
Now let us add series connections from left to right
and parallel connections from top to bottom.
A 3 layer neural network
Hey presto! We have a 3 layer neural network!
The output of every neuron is connected to every
neuron in the corresponding next layer. This will
provide the possibility of realizing every probable
input output relationship, so long as there are
enough layers of neurons in the hidden layer.
But Wait! Before we can implement any system
using this network, we need to have the
appropriate weights for that system.
Going back to Mother Nature for inspiration;
Human beings learn by experience. Similarly, we
can have a process of training the ANNs, to
achieve the desired weights.
This would be similar as solving a puzzle of
„Complete the Series‟
Here, in the first case, the input is 2 and the output
is 4. On observing this, we try to find out the
relationship between input and the output. Now, 2 2
is 4, 2+2 is 4, 2 x 2 is 4, and so on… Then we
move on to the next set of data, 4 and 6. After
looking at such sets of data, it finally becomes
clear that +2 is the relationship which satisfies all
the examples. From this relationship it is
established that the output is 10.
Similarly, while training, ANNs should be
provided with sets of inputs and outputs known as
“training examples”. Then, an iterative algorithm
be adopted to modify the weights, to obtain values
of weights which satisfy all examples. Thus
establishing a generalized relationship which
would produce the correct output in all possible
conditions which could occur during operation.
Because we use practical examples for training, the
relationship developed is free from the
assumptions and approximations taken to
formulate the mathematical model of a system, as
in conventional systems. Thus the relationship
developed is very accurate.
However, to achieve this we need to cover all types
of data during the training. Thus, training becomes
a very complicated and time consuming process.
However, training is only a one time issue.
But, how do we realize ANNs?
We saw the mathematical model of ANNs. The
mathematical functions can be realized using op-
amps, and thus the ANNs. We may also map the
entire circuitry on a single chip using VLSI
technology.
However, we do not have any way to predict
beforehand, what kind of a neural network, how
many layers, what kind of training, etc. will be best
suitable for the desired application.
So, for the purpose of development, testing and
simulation, tools like MATLAB and LISP can be
used to realize ANNs superficially. Though, in
some cases these may be used as the mode of
realization for operation too.
Although we‟ve managed to invent ANNs here, it
is not difficult to guess that the ANN technology is
a pre-existing one. Let‟s now take a comparison of
ANNs with conventional systems.
ANNs v/s Conventional systems
ANNs are able to realize systems having implicit
input output relationships, which conventional
systems can‟t.
Example: In the case of launching a rocket,
various laws of physics are involved. These laws
are expressed mathematically, as a result of which
the relationship becomes clearly definable. Such
relationships are easily realizable using
conventional systems.
Conversely, in cases like, face detection in an
image, obviously no such mathematical laws are
applicable, and thus the relationship can‟t be
clearly defined. But, ANNs are able to realize such
relationships, due to the method of design used
(involving training).
In conventional systems, we have serial
computation i.e.; only 1 thing is processed at a
time, but at a high speed. So, even if a single
activity gets disturbed the entire output gets
degraded.
In ANN based systems, we have highly parallel
computation i.e.; at a time different processing is
being done by a large number of neurons. So, even
if degradation happens, it is graceful, because each
neuron has only a small part to play.
Like, conventional systems require programming,
Neural Network based systems require training
before they can be put to work.
Areas of Application
ANNs are used in systems where thinking and
analytical skills are required; from a broader
perspective, they may be used in any system where
the relationship is implicit.
Due to the accuracy of ANNs in deciphering the
relationships, they are also quite commonly used to
devise the formula while designing systems.
Conclusion
Artificial Neural Networks are adaptive in nature
and are hence more versatile.
With their versatility and the ability to provide
control customized for specific application, ANNs
are set to break all the barriers in automation; those
which exist due to the inabilities of present day
conventional systems, to process like the human
brain.
- Prashant Serai Shuchi Jain
(II Year EXTC)