ST.

THOMAS’ COLLEGE OF ENGINEERING &

TECHNOLOGY

Name: Rohit Sardar

Dept.: ECE
Year: 4th
Roll No.: 57
Subject: Neural Networks & Applications

Digit Recognition System using the

Backpropagation Model
INTRODUCTION
Digit recognition is an art of detecting segmenting and identifying digit from image.
The phenomenon of digit recognition involves detection and recognition of digits
from input image and then it is converted into ASCII or other equivalent machine
editable form so that it can be processed. It helps greatly to the advancement of
automation process and improving the interface between man and machine in many
applications. Digit Recognition system is the combination of pattern recognition and
artificial neural network. Digit recognition is getting more and more attention since
last decade due to its wide range of application. This is because it has an importance
in several fields and it may probably be used in checks in banks or for recognizing
numbers in cars plates, or many other applications. Conversion of handwritten digits
is important for making several important documents related to our history, such as
manuscripts, into machine editable form so that it can be easily accessed and press
independent work is going on. Digit recognition system can be implied by both the
techniques i.e. offline as well as online. Both the techniques have their advantages as
well as constraints. Offline digit recognition system, in this document is first generated,
digitized, stored in computer and then it is processed. While in case of online digit
recognition system, digit is processed while it was under creation. In offline system
external factors like pressure, speed of writing has little influence, but they have great
impact on online system. Again, offline or online system (Fig 1. (a) & (b)) or
handwritten digits can be applied on optical digit (Fig 2.(a)) or handwritten digits (Fig
2.(b)).
Back Propagation Network
Learning
Back propagation (BP) learning is a supervised learning method. In such type of
learning methods, the network is first trained with a training set that has fixed input
patters and the output for each of the patterns is predefined. This helps the network
to learn and adjust the weights accordingly before the actual inputs are applied to the
network. The network is tested once its fully trained using a testing/validation set. BP
learning is simply adjusting the weights of the system in direct proportion to the
error.[4][5] The error function used in BPA is Mean Square Error (MSE) given as,

Architecture
Back propagation Network Architecture The typical BP network consists of three
layers, input layer, hidden layer and output layer which can be seen in fig. 9. The bold
lines in the figure denote the forward path which is propagation of the normal inputs
in the network while the dotted lines denote the backward path which is propagation
of the error

Back propagation Network Architecture The typical BP network consists of three

layers, input layer, hidden layer and output layer which can be seen in fig. 4. The bold
lines in the figure denote the forward path which is propagation of the normal inputs
in the network while the dotted lines denote the backward path which is propagation
of the error.
 Fig 4: Typical Backpropagation Neural Network

Algorithm
BPA is Generalized Gradient Descent i.e. generalization of LMS Algorithm. BPA uses
this generalized delta rule to adjust weights by calculating current error and then
backpropagating this error layer by layer.[4][5]
The algorithm is mentioned below:

 Initialize all weights and biases to some small random values from [-1,1].
Repeat below mentioned steps till network converges:
 Handwritten Digit Recognition Using Back Propagation Neural Network

i. For each training pattern presented in random order. Apply the inputs to the network

ii. Calculate the output for every neuron from the input layer, through the hidden
layer(s), to the output layer.
iii. Calculate the error at the outputs
iv. Use the output error to compute error signals for pre-output layers.
v. Use the error signals to compute weight adjustments

vi. Apply the weight adjustments

vii. Periodically evaluate the network performance

Generally, a neural network model prearranges the number of layers in advance. A BP

neural network may include a number of hidden layers. However, it has been proved
in theory that a 3-layer BP neural network may achieve discretional non-linear
mapping without limiting the number of nodes in the hidden layer. Thus, the BP neural
network model used in this paper uses only three layers. The design and
implementation of the BP neural network is mentioned below:
MATLAB Implementation
function[weight1,weight2]=backPropagation(trainSet,trainclassLabel,neurons,mome
ntum,trainSize)
%Initializing weight matrix with random values and dividing them by 10
%so the most of the values fall between [-1,1]
weight1 = randn(neurons,784)/10; weight2 = randn(10,neurons)/10;
outputAtHidden = zeros(1,neurons); outputAtOutput = zeros(1,10);
covergence = 0; dconvergence = 0; Nweight = 0; Ndweight = 0;
variable = 1;itre = 1;
while(1)
%Building the class label
label = [0,0,0,0,0,0,0,0,0,0];
temp = trainclassLabel(itre) + 1;
label(temp) = 1;
%---------------------------------
%Variable Learning Rate
alpha = double(1/sqrt(variable));
%Feed Forward =
[outputAtHidden,outputAtOutput]=feedForward(trainSet(itre,:),outputAtHidd
en,outputAtOutput,weight1, weight2);
%Memorizing previous weights to calculate the change in error later
%playing as a stopping criteria
Nweight = Ndweight;
%Squared Error Function
Ndweight = (1/2)*(label-outputAtOutput)*(label-outputAtOutput)';
%Calculating Error at output layer
errorAtOutput = (-1)*((label-
outputAtOutput).*outputAtOutput.*(ones(1,10)-outputAtOutput));
%Updating weights between hidden layer and output layer
weight2 = weight2 - alpha*(errorAtOutput'*outputAtHidden);
%Calculating Error at hidden layer
errorAtHidden = (errorAtOutput*weight2).*(outputAtHidden.*(ones(1,100)-
outputAtHidden));
%Updating weights between input layer and hidden layer
 weight1 = weight1 - alpha*(errorAtHidden' * trainSet(itre,:));
%Calculating covergence of error
covergence = covergence + abs(Ndweight-Nweight);
%After one complete iteration, we check whether the total error is
%converged upto a certain limit to break out of the loop
if itre == trainSize
fprintf('')
fprintf(sprintf('Status Update 1, %i iterations done on
stoppingCriteria %d\n',variable/(54210),momentum));
fprintf(sprintf('Status Update 2, convergance difference :
%f\n',abs(covergence-dconvergence)));
if abs(covergence-dconvergence) < momentum
break;
end
dconvergence = covergence;
covergence = 0;
end
itre = mod(itre, 54210)+1;
variable = variable+1;
end
end