Arti cial Neural Networks..

Notes
Kriti Agarwal
March 5, 2011

01Introduction
Arti cial Neural Networks are an information processing technology
inspired by the studies of the brain and the nervous system.

ANN's is either supplementing or replacing statistical and

conventional expert systems in nancial decision making.
ANN emulates the biological neural network.

02Components and Structures

Processing Elements (PE): Ariti cal neurons are called the Process-
ing Elements. Each PE receives input(s), processes the input(s)
and produces a single output.

Network: All the PE's are grouped into layers.

Network Structure: ANN can be organized into several topologies.

Processing of the Data in the network: The major elements

involved in processing are inputs, outputs and weights.

{ Inputs: Each input corresponds to a single attribute. e.g. if the

problem is to decide the approval/disapproval of a loan the at-
tribute could be income level, age or ownership of a house etc..
Value of input could be numbers or qualitative inputs like "yes/no".

1
 { Outputs: Output of the network is the solution.. either a "yes" or
"no" in the above example of loan.
{ Weights: Key element of ANN. It represents the relative
strength of the input data or connection between di erent
layers. It is through the repeated correction of the weights that
the ANN learns.

Summation function: weighted average of the inputs elements of

PE. i.e sum of the product of each input and weights. It is also
called the Activation function

Transfomation function: The output of the summation function is called

the activation level. The Transformation function gives the re-
lationship between this level and the output. e.g. Sigmoid function
1
Yt = 1 + e y (1)

A transformation function can occur at the output of each PE or at

the output of the network.
e.g.

x1 = 3; x2 = 1; x3 = 2
w1 = 0:2; w2 = 0:4; w3 = 0:1
3(0:2) + 1(0:4) + 2(0:1) =
y = 1:2
1
Yt = 1 + e 1:2 = 0:77
Learning: In simple terms it involves three steps.
1. Compute outputs
2. Compare the output with the desired output.
3. Adjust the weights and repeat the process.

ANN Strengths and Weakness

Fault Tolerance - damage to one or two nodes doesnt have a huge
im-pact on the network.
Generalization - unknown input.. it still gives a reasonable
input. Adaptability - Learning..

2
Works well only on data with structure similar to the training set
Most results lack justi cation - e.g. pattern recognition.
large amount of data and lengthy training times.
Cannot guarantee optimal solution.. sometimes for the same input
it can give two di erent outputs at di erent occations
Application
Credit Authorization Screening
Mortgage risk assesment
Project management and bidding
strategy Financial and economic
forecasting. Prediction of default and
bankruptcy Portfolio selection
Simulation of market behaviour

Developing Neural Network

Applications. Step 1: Collecting data
Step 2: Seperating it into training set and test set

-thorough analysis of the application - bounded problem -fully

under-stand the context of the application and use of neural
network -seek help of domain expert

-minimize ambiguity in the collected data - data should cover the

widest range of problem domain - should cover routine operation as
well as exceptions - better quality data implies the faster
convergence to the weights.

Network Structures
(1) Associative Memory Systems - Associative memory is
the ability to recall the complete situation from partial
information. These systems correlate input data with
stored data in the memory. e.g. Hop eld network.
(2) Hidden layer - Associative Memory System can have
one or more intermediate layers
(3) Double Layer Structure - A little complex.. if needed
will come back to this topic.. look at the gures in the
text..

3
Learning Algorithms
Two categories - Supervised Learning and Unsupervised Learning
(1) Supervised Learning uses a set of inputs for which
the appropriate outputs (desired outputs) are known.
The di erence between the desired outputs and actual output is
used to correct the weights.
e.g. Hop eld Network
(2) Unsupervised Learning - Self organizing .. only the
input is given to the network... a human must examine
the nal categories and assign meanings and
determine the usefullness of the output..
e.g. Kohonen Self-organizing maps

How a network learns? example in the text page 16 and 17.. good
example..
Training the network
Training data is given to the network.
Each data vector is presented to the network and the weights
adjusted till a consistend output is obtained.
The structure of the network and the initial values chosen will deter-
mine the time taken by the network to converge.

few topics which are more related to house keeping... will come
back to them
Topics are related to Implementation and convincing management.

Neural Computing Paradigms

Size of the training and test
data Learning algorithm to be
used Topology to be used
Learning rate for each
layer Validation tools.
Software and Hardware
ANNs are software applications. And can be programmed by a pro-
gramming languager or tool or both.
Major portion of the programming deals with the training,
summation and functions.
Faster machines - Neural chips - Accelerator boards

4
||||End of Chapter 1|||||-

Decision Environment
Managerial Decisions classification
| {z }
Structured and Unstructured
Structured tasks can be handled using basic computer systems,
comer-cial software and low level management.
But the decisions to be taken by the top managers are often not
struc-tured..