ARTIFICIAL INTELLIGENCE

GENESIS
• An old dream
– To produce an intelligent replica of one’s own
– From Merry Shelley’s Frankestine to Stephen
Spilberg’s summer movie AI the dream has cast
its shadow
 Some Important Observations
• Intelligence is the key to progress of
mankind.
• Power of intelligence lies in the knowledge
• Knowledge is a product of human
cognition/thought processes
 Philosophers’ Endeavour
• For more than 2000 years, philosophers are
trying to reveal the mysteries of human
cognition
 A Valuable Outcome
• The Hypothesis
– Human mind is principally governed by
reasoning with knowledge contained in it.
 The Idea
• Intelligence can be capture by doing the
following.
– Defining Knowledge
– Mastering the mechanism of reasoning
analytically
 Materialization of the Idea
• Past efforts on how to capture intelligence
have led to the development of the subject
of “Artificial Intelligence”
 Three Popular Examples
• Deep Blue
• Humanoid Robots
 Three Popular Examples
• Deep Blue
– It is an IBM made chess playing system, which defeated
the world champion Gary Kasparov in 1997
• Humanoid Robots
– They are often found dancing, singing and even playing
soccer.
• Every year, a soccer tournament for robots are organized in
Japan under the auspices of RoboCup
• Robocup’s ultimate goal is to beat humans in soccer by 2050
• Many AI professionals consider RoboCup small league
competition as an ideal platform for testing distributed AI
techniques
 Three Popular Examples
contd..
• Autonomous Vehicles
– A system called Autonomous Land Vehicles in a
Neural Network (ALVINN) was developed in CMU in
early 1990’s
– It is capable of steering automobiles autonomously
– It successfully drove a van from Washington DC to San
Diego.
– During this journey, it has covered a distance of all but
52 of 2,849 miles at an average speed of 63 mph over
day and night, under all weather conditions.
 Artificial Intelligence
• It is a subject that attempts to understand the
mechanism of intelligence and strives for
computational modeling of intelligent behaviour
• Intelligent behaviour involves the following
– Perception
– Reasoning
– Learning
– Communicating
– Acting in a complex environment
 A Grand Question
• Whether artificial consciousness can really emerge in
advanced computers?
– The belief that all important aspects of human cognition
may in principle be captured computationally forms the
very basis of AI
– The belief has made it possible for AI people to focus on
the abstract causal structure that the mind embodies (at an
appropriate level) instead of the biological substrate of
the mind, i.e., neurons and biological wetware in general
– According to this belief, formation of best possible minds,
i.e., human minds, on a carbon based substrate is possible
due to constraints of evolution and possibly historical
accidents, rather than to absolute metaphysical necessity
 A Grand Question
• The view is made in two steps
Step 1: Church-Turing Thesis supports the idea that any
abstract causal structure can be modeled computationally
Step 2: Since publication of an article by Alan Turing,
supporting the idea for making the computer intelligent, in
1950, AI people started to believe the possibility of viewing
human minds through abstract causal structures
• Viewing human minds through abstract causal
structures has become known as functationalism
– After Turing (1950), it was explicitly articulated in various
forms by Putnam (1960), Armstrong (1970) and Lewis
(1970)
 A Grand Question
• Viewing computational modeling as essential for
capturing essence of cognition is called
computationalism
• The Church-Turing Thesis has bridged the gap
between functionalism and computationalism
• Some believe in fnctionalism but not in
computationalism, but there are many people who
believe that two are synonymous
– Penrose (1989), who seems to be a functionalist without
being a computationalist, believes that certain causal
properties of the mind cannot be captured algorithmically
 A Grand Question
• Philosopher John Searle has termed the
doctrine of computationalism as ‘strong AI’
– In other words, the belief that a computer
system equipped with sufficient processing
power and sufficiently powerful artificial
intelligence would actually be capable of
having mental states in the humans is called
strong AI.
 A Grand Question
• In contrast to ‘weak AI’, holding that the computer
can be used as a powerful tool for understanding
the mind, strong AI goes far beyond holding that
the appropriately programmed computer really has
(is) a mind
– Weak methods in AI attempt for a general purpose
search mechanism trying to string together elementary
reasoning steps to find complete solutions.
– Such methods that had arisen in the first decade of AI
research use weak information about the domain
 A Grand Question
• The concept of strong AI is severely criticized by Searle in
“Minds, Brains, and Programs” (1980)
• He argues through his now famous Chinese Room argument
that a computer can never recreate such vital properties of
human mentality as internationality, subjectivity, and
understanding
• In his view, strong AI is thoroughly implausible.
• Later David J. Chalmers in “Subsymbolic Computation and
Chinese Room” (1992) observes that Searle’s arguments
provide at least a plausible criticism of the Symbolic
approach to AI, but connectionist models are less vulnerable
to his arguments
– He advised Searle to desist from his blanket criticism of the entire
AI research endeavor and to think over the potential of
Connectionist approach to AI.
 Major Goals of AI
• The Long Term Goal
– To develop a general information processing theory of
intelligence
• The Short Term Goal
– To realize the intelligent abilities, such as the
following, through computers
• Theorem proving
• Program synthesis
• Expert decision making
• Game playing
• Language comprehension
• Commonsense reasoning
 Dartmouth College Workshop
• It was an international workshop held in 1956
• How every aspect of learning or any other feature
of intelligence can be realized with the computer
was the theme of the workshop.
• To express the theme, John McCarthy coined the
term “Artificial Intelligence”
• The event was a land mark for emergence of AI as
a subject of study first time in an international
forum.
 Dartmouth College Workshop
• Some earlier developments that necessitate
holding of one such workshop are the
following
– Commercial use of stored program computers
– Alan Turing’s arguments supporting the idea of
programming a computer to exhibit intelligent
behaviour
– The work of Claude Shannon on the possibility
of computer chess
 Dartmouth College Workshop
• Development of the first neural network
computer by Marvin Minsky and Dean
Edmonds
• The progress of the work by Alan Newell
and Herbert Simon on automatic theorem
proving
• Noam Chomsky’s work on the theory of
generative grammars
 Turing’s Article
• The hope that machines would eventually
compete with human was expressed in the
article in 1950
• Turing argued for the idea of programming
a computer to exhibit intelligence
• He proposed a test to judge the intelligence
of a machine
 Turing Test
• A human interrogator
sitting in a room asks
a series of questions
• The questions are A computer
randomly directed to
either a computer or a
person in a separate A human
interrogator
room.
A person
 Turing Test
• If the interrogator is
not able to distinguish
which of the two has
responded to his A computer
queries then the
computer will be
called intelligent A human
interrogator

A person
 Deep Blue and Turing Test
• Deep Blue seems to have passed Turing
Test
– Kasparov could never wholeheartedly accept
that he played with a computer itself
 Secrets of Deep Blue’s Feats

• Ability for high speed computation

• Intelligent Search Strategies
• Innovative Partitioning of Search Problem
 Secrets of Deep Blue’s Feats
• Ability for high speed computation
– Major number crunching calculations are
assigned to ASICs
– ASICs are incorporated into the processors
forming a multiprocessor configuration
– Deep Blue is capable of analyzing 200 million
positions per second
 Secrets of Deep Blue’s Feats
• Intelligent Search Strategies
– It is needed to manage the complexity of the entire search
space of 10120 board positions
– Deep Blue would require 1095 billion years to explore the
entire space
– Search for the best move is restricted to an analysis of the
most promising positions
– Expert knowledge is used to perform an optimized game
tree search
– Combination of speed and the smart search algorithm
gives Deep Blue both positional and material advantages
in matches
• Innovative Partitioning of Search Problem
– It plays a great role to maintain Deep Blue’s hardware
software synergy
 Intelligent Computation
• It is needed for machine realization of intelligent
behaviour
• It does not necessarily mean exact modeling of human
thought processes and reasoning mechanism
• It is better to refer present day AI as intelligent
computation
• The following are some of the areas where computers
have begun to approach human level intelligence
– Continuous Speech Understanding
– Electro Cardiogram (ECG) Diagnosis
– Theorem Proving
– Aircraft Guidance
 Commercial Significances of AI
Technologies

• Improved Industrial Productivity

• High cost saving
 Some Industrial Products
• XSEL, a DEC made sales support expert system
• Optex, a camera lens design system
• An authorizer’s assistance system used by
American Express to evaluate credit proposals
• SMART system for diagnosing Compaq products
• Prism Telex Classification system, Developed by
Cognitive System
 XSEL
• It is a sales support expert system, developed by
DEC
• It reduces a three hour system configuration
alternative generation task to fifteen minutes.
• It also reduces the number of non-manufacturable
system specifications from 30% to 1%
• All this is worth $70 million per year
 Optex
• It is a camera lens design system
• It increases productivity by 12 times for the
scarce, highly skilled lens design system
 An Authorizer’s Assistance System

• It is used by American Express to evaluate

credit proposals
• It reduces the decisions to deny credit to
customers by one-third
 SMART System
• It receives diagnostic problems arising from
the use of Compaq products and makes
recommendations about necessary services
• It raises the percentage of customer
problems resolved on the first call from
50% to 87%
 Prism Telex Classification System
• It is developed by Cognitive System
• It is used to route incoming telex
communications to appropriate recipients
• It has raised classification accuracy from
75% to 90%
• It saves the average time to route a telex
from several minutes, previously required
for a telex operator, to 30 seconds
 Two Major Approaches to AI

• Symbol Processing Approach

• Sub Symbolic Approaches
– Animat Approah
– Connectionist Approach (also called Artificial
Neural Network Based Approach)
– Evolutionary Approach
 Symbol Processing Approach
• Symbols constitute A B
basic building blocks D C

‘ABCD is a rectangle’
– All real world objects
and concepts are RECTANGLE(ABCD)
represented with
symbols Concept Object
‘Socrates is a man’

MAN(SOCRATES)

Concept Object
 Symbol Processing Approach
• Symbols may be
combined to form
complex structured ‘All rectangles are parallelograms’
expressions
(x) RECTANGLE(x) PARALLELOGRAM (x)
• Complex expressions
refer to more complex ‘Man is mortal’
entities or knowledge
(x) MAN(X) => MORTAL(x)
• All kinds of data that can
improve efficiency and
effectiveness of a
problem solver,such as
facts, beliefs, and rules of
thumb(heuristics), are
considered as knowledge
 Symbol Processing Approach
• Under this approach, computation is performed directly upon
symbols to replicate intelligent behaviour, i.e., to generate
inferences
• Inferencing is realized with a class of specially designed programs
– The programs can compute new symbols from the existing ones,
e.g.,
MAN(SOCRATES)
(x)MAN(x) => MORTAL(x)

MORTAL(SOCRATES) {SOCRATES/x}

RECTANGLE(ABCD)
(x) RECTANGLE(x) PARALLELOGRAM (x)

PARALLELOGRAM (ABCD) {ABCD/x}

A Complex Chain of Inferences: An Example

• Consider the following sentences:

1. Any two who are class mates are friends.
2. If x and z are classmates and z and y are friends then x and
y are friends.
3. Ram and Rahim are classmates.
4. Rahim and Ali are classmates.

Prove that Ram and Ali are friends.

A Complex Chain of Inferences: An Example
contd..
• Goal Expression: FRIEND(RAM,ALI)
The pieces of knowledge supplied intially:
1.(u) (v)CLASS-MATE(u,v)FRIEND(u,v)

2.(x) (y) (z){[CLASS-MATE(x,y)ΛFRIEND(y,z)]FRIEND(x,z)}

3. CLASS-MATE(RAM,RAHIM)

4. CLASS-MATE(RAHIM,ALI)
A Complex Chain of Inferences: An Example
contd..

• Proof:
CLASS-MATE(RAHIM,ALI)
(u) (v)CLASS-MATE(u,v)FRIEND(u,v)
FRIEND(RAHIM,ALI) {RAHIM/u, ALI/v}

CLASS-MATE(RAM,RAHIM)
(x) (y) (z){[CLASS-MATE(x,y)ΛFRIEND(y,z)]FRIEND(x,z)}

FRIEND(RAM,ALI) {RAM/x,RAHIM/y,ALI/z}
The Physical Symbol System Hypothesis

• A physical symbol system has the necessary and

sufficient means for intelligent actions [Newell and
Simon, 1976]
• An important aspect of the hypothesis is that the
physical symbol system is substrate neutral
– It does not matter what the physical symbol system is
made of !
– An intelligent entity could be made of protein,
mechanical relays, transistors, or anything else, so long
as it can process symbols
 The Knowledge Based System (KBS)

• It is a domain specific problem solver,

designed on the principles of Symbol
Processing Approach.
• Since it can work like a human expert in a
specific application domain, it is also known
as the expert system.
 Major Components of A KBS
• Two Major Components
– A Knowledge Base (KB)
– An Inference Engine
• An Additional Component
– A Learning Module
 Learn-
Inference ing
Know- Teacher
Engine Module
ledge
User Base

Training
data

Knowledge
Engineer
Domain Expert

A BLOCK DIAGRAM OF A TYPICAL KNOWLEDGE BASED SYSTEM

 Major Components of A KBS
• A knowledge base (KB) consists of the following:-
– Facts about the problem
– The rules governing the relations among the various
entities of the problem domain.
• An inference Engine(IE) is characterized as follows :-
– It is essentially a computer program.
– It specifies the logical process, by which new facts and
beliefs are derived from the existing ones in the KB.
– It also embodies a control strategy that manages the
search for an inferential solution.
Conclusion on Symbol Processing Approach

• The description of a mental process at the symbol

level is necessarily a description at the conceptual
level, as atomic symbols are by definition
representation of concepts
• According to Physical Symbol System hypothesis,
all mental processes can be described by
computation on atomic symbols
• Mostly top-down design methodologies are used,
beginning at knowledge level and proceeding
downward through the symbols and
implementation levels
 Sub Symbolic Approaches
• The Key Concept
– To model intelligent behaviour, processes working
below the symbol level, i.e., at the signal level, must be
considered
• Smolenesky’s Sub Symbolic Hypothesis (1988)
– The intuitive processor is a subconceptual connectionist
dynamical system that does not admit a complete,
formal and precise conceptual level of description
 Animat Approach

• It was developed for advancing both robotics

and AI by building an autonomous agent
• It does not assume the existence of any central
control that does planning with the
representation of the world
 Animat Approach

• It is based on the following idea

– In order to make intelligent machines, one has
to follow the evolutionary phases, through
which human intelligence has come to its
present form, being enriched after every phase
for billion of years
• Development of intelligence through evolutionary
phases is an incremental procedure, i.e., intelligence
of lower animals has appeared in an improved form
in higher animals, after every phase of evolution
 Animat Approach

• Developing an intelligent machine through

evolution based incremental procedure
involves the following
– Develop a substrate duplicating sensing and
action performing abilities of insects like lower
animals
– Augment the substrate incrementally with
program modules replicating higher level
intelligent abilities through a process similar to
evolutionary phases
 An Illustration of Animat Approach

• Consider the design of a mobile robot to

move on a floor avoiding obstacles
– The following is the block diagram of one such
robot designed under Symbol Processing
Approach
– The design is based on a central control that
does planning with the world representation
around the robot
Signals Perception Motor
from and Control Signal
sensors Planning Control
Modeling generator signals

The model developed under symbol processing approach

• Perception and Modeling involve

capturing images of the surrounding
environment and identifying objects therein
• Planning involves inferencing
An Illustration of Animat Approach
– To design the robot through Animat Approach, the
following simple behavior generating modules that enable
the robot to act are designed first
• Avoid module letting the robot to percept only to avoid obstacles
• Wander module generating a random heading for the robot at
regular intervals of time
• Explore module letting the robot to percept only to visit distant
visible places
– All the modules work in parallel, i.e., there is no central
control
– For the final action, combination of actions of all
individual behavior generating modules is necessary
 Explore
Module
Inhibit Motor
Signals
from control
Wander signals
sensors
Module

Avoid
Module
Final output

Fig: The block diagram of a mobile robot designed through Animat

approach
 Subsumption Architecture (Brooks
1986,1990, Connel 1990)

• Interconnections of layered behaviour generating

modules designed under Animat approach result
into an architecture called the Subsumption
Architecture
• In this architecture, the goal of each layer is
subsumed by the goal of its immediate higher
layer.
– For example, the functions of the Avoid module remain
operative within the functions of the Wander module of
its immediate higher layer
 Connectionist Approach
• It attempts to model the functions of nerve cells or
neurons and their interactions inside the human
brain, for replicating intelligent behaviour.
• It aims to replicate especially learning and
generalization abilities of human intelligence with
various models of neural networks.
– The model of the biological neural networks so
produced are called artificial neural networks(ANNs)
 Connectionist Approach
• Modeling the functions of the human brain is not an easy task.
– The human brain computes in an entirely different way
from the conventional digital computer
– It is a highly complex, nonlinear, and parallel system
– It has the capability to organize neurons, i.e., its structural
constituents, so as to perform certain computations many
times faster than the fastest digital computer in existence
today
• The brain routinely accomplishes perceptual tasks (e.g.,
recognizing a familiar face embedded in an unfamiliar scene) in
approximately 100-200 ms, whereas tasks of much lesser
complexity may takes days on a conventional computer
– The computations involve
• Pattern recognition
• Perception
• Motor control
 Major Features of Biological Neural Networks

• A neuron or nerve cell is a unit of biological neural

networks
• The human brain contains approximately 1014 to 1015
neurons, which is 1000 to 10,000 times larger than the
number stars in the Milky Way
– The number of neurons in the cerebral cortex is nearly 1011
– The cerebral cortex in human is a large flat sheet of neurons
with a thickness of 2 to 3 millimeters and a surface area of
about 2,200 cm2, about twice the area of a standard computer
keyboard
• Each neuron is connected to 10 3 to 10 4 other neurons
 Major Features of Biological Neural Networks

• The following are the three major parts of a neuron

– The Cell Body or Soma
• It has a nucleus that contains information about hereditary
traits and a plasma that holds the molecular equipment for
producing materials needed by a neuron
– The Dendrites
• A neuron receives signals (impulses) from other neurons
through its dendrites (receivers)
– The Axon
• A neuron transmits signal generated by its cell body along the
axon (transmitter)
• The axon eventually branches into strands and sub strands
 Major Features of Biological Neural Networks

• The operational speed of a network of biological neurons is only

a few milliseconds
– Neurons communicate through a very short train of pulses,
typically milliseconds in duration
– The message is modulated on the pulse- transmission
frequency, varying from a few to several hundred hertz
• It is a million times slower than the fastest switching
speed in electronic circuits
• With this speed, complex perceptual decisions such as face
recognition typically require a few hundred milliseconds for
humans
• This implies that the computations cannot take more than 100
serial stages
 Major Features of Biological Neural Networks

• The brain runs parallel programs that are about

100 steps long for face recognition like perceptual
tasks
– This is known as the hundred step rule
• The same timing considerations show that the
amount of information sent from one neuron to
another must be very small (a few bits)
• This implies that critical information is not
transmitted directly, but captured and distributed
in the interconnections-hence the name
connectionist model used to describe ANN
 Major Features of Biological Neural Networks

• Neurons are connected through synapses in biological neural networks

– A synapse is an elementary structure and functional unit between two
neurons, i.e., between an axon strand of one neuron and a dendrite of
another neuron
– It acts as a gate way for signals between any two neighboring
neurons connected in a biological network
– It releases certain chemicals called neurotransmitters sensing
presence of signals at its terminal
– Diffused across the synaptic gap, neurotransmitters enhance or
inhibit, depending on the type of the synapse, the receptor neuron’s
own tendency to emit electrical impulses (signals)
– The synapse’s effectiveness can be adjusted by the signals passing
through it
• Since similar signals cause similar adjustments, synapses can learn from the
activities they participate
• Dependence of the synapses’ effectiveness on history possibly works behind
the functions of human memory
 Major Features of Neuron Level Learning

• The following are the major features of neuron level learning

mechanism so far discovered
– Actions of biological neural networks are guided by
controlling the tendencies of neurons to emit bio-electric
signals
– When bio-electric signals reach a synaptic junction, certain
chemicals called neurotransmitters are released there
• Neurotransmitters there work either to inhibit or exhibit,
depending on the type of the synapse, the receptor
neuron’s own tendency for emitting bio-electric signals
– Synapses effectiveness can be tuned to produce similar
responses under similar situations
– Dependence of the synapses’ effectiveness on history is
possibly responsible for human memory functions
 Major Features of ANNs
• A biological neuron is modeled as a computational
node in ANNs
• The effectiveness of a synaptic junction is modeled by
assigning appropriate connection weights to all
incoming links to the junction
• Each computational node performs an weighted sum of
all its incoming signals
– It fires by producing a high valued output signal if the sum
exceed certain threshold
• An ANN requires training by some suitable learning
algorithm to learn to work effectively with all possible
problem instances
 Major Features of ANNs
• Architectural and functional features of ANNs
make them fit for offering the following to various
applications
– Massive Parallelism
– Distributed Representation and Computation
– Fault Tolerance
• ANNs are also called connectionist models as the
critical information in these models are captured
and distributed in the interconnections
 McCulloch-Pitts (MCP) Model of a Neuron
• It is one of the oldest models of biological neurons, proposed
by McCulloch and Pitts in 1943
• Since it performs binary thresholding on the sum of its
weighted inputs it is also known as the Threshold Logic
Neuron (TLN)
• It is defined by the following:
– The number of inputs (n)
– The weight values (w1, w2, …, wn) associated with the
inputs
– The threshold value (u) associated with the output (y)
– Supplied with input signals (x1, x2, …, xn), the output (y) of
the MCP neuron becomes

– Where is a unit step function at u, and wi is the synapse

weight associated with i th unit
 McCulloch-Pitts (MCP) Model of a Neuron

– For simplicity the threshold u is considered as another

weight w0=-u attached to the neuron with a constant input
x0=1. With these,

1 if >0
y=
0 Otherwise
– Positive weights correspond to excitatory synapse
– Negative weights model inhibitory ones
– Threshold function approximates the activity in soma
– Wires and interconnections model axons and dendrites
– The simplifying assumptions for the MCP model do not
reflect the true behavior of biological neurons
 x1

w1

1 y
x2 w2

.
. u
.
wn

xn

McCulloch-Pitts model of a neuron

 McCulloch-Pitts (MCP) Model of a Neuron

• McCulloch and Pitts proved the following

– In principle, suitably chosen weights let a synchronous
arrangement of such neurons performs universal
computations
 Generalizations of the MCP Neurons

• The MCP neuron is generalized in many

ways by using the following functions in
place of the threshold function
 Generalizations of the MCP Neurons

• The sigmoid function is by far most

frequently used in ANNs
– It is a strictly increasing function exhibiting
smoothness
– It has the desired asymptotic properties
– The standard sigmoid is the logistic function
defined by
g(x)=1/(1+e-ßx) ,
where ß is the slope parameter
 Generation of a Hyper Plane by the MCP Neuron
• Every MCP neuron generates a hyper plane
dividing the input space in two parts
– For a two input MCP neuron it is a straight line
w1x1+w2x2+w0=0

X2

x2=(-w1/w2)x1+(-w0/w2)

(0,0) X1
Hyper plane
A straight line dividing the input plane
 Generation of a Hyper Plane by the MCP Neuron

• Inputs from the opposite side of the hyper

plane generate complimentary responses
with the MCP neuron
 An MCP Neuron as a Pattern Dichotomizer
• For machine recognition of patterns, they are to be
represented as points or vectors in the feature space
– Patterns are represented as feature vectors by measuring values
of a fixed number of suitably chosen features from their digital
images
– For example, the patterns of handwritten English digits can be
represented with the two features
• The lengths of projections on two horizontal sides of the bounding box of
each digit image, taken after horizontally dividing the box into two equal
halves, known as the shadow features
– With these two features, handwritten sample images of digit
patterns ‘4’ and ‘9’ will form two separately spaced pattern
clusters in the feature space
– By finding an equation to a straight line separating the two
clusters in the feature space, any unknown pattern can be
classified in either of the two classes
 f1
f1

f2 f2

Shadow features extracted Shadow features extracted

from a digit image of ‘4’ from a digit image of ‘9’

An illustration of simple shadow features

 f2

44444444
444444 444
4 44 4 44 44
44444444
44444 4
4 4
9999999999
9 9 9 99 9 9 9 9 9 9
99 99 9999 99
99 99 9 9999
9 9 9 99
9
f1

Two cluster of patterns generated by representing

sample images of ‘4’ and ‘9’ with shadow features f1 and
f2
 An MCP Neuron as a Pattern Dichotomizer

• The problem of pattern recognition can be solved

through mapping measurement space (M) to
feature space (F) and from feature space to
decision space (D) as follows
M F D
• Mapping feature space to decision space
requires a suitable discriminant function
 An MCP Neuron as a Pattern Dichotomizer

• For two linearly separable pattern classes, a suitable

discriminant function can be generated by proper
tuning of the neural weights of an MCP neuron
through a method called learning
• It is possible because the hyper surface
generated by each MCP neuron can act as the
discriminant function required for pattern
recognition
• By virtue of this property an MCP neuron can act as a
pattern dichotomizer generating two decision regions
in the pattern space
• The technique will not work for two nonlinearly
separable pattern classes
 c1 c1 c2
f2 f2
++ ++ --
++ ++ -
++ ++ ---
++ -- -
-- -- -
++
-- c2 -
++
-- -- -
-

f1
f1
Two linearly separable
Two nonlinearly separable
pattern classes
pattern classes
 Recognition of AND Inputs using an MCP Neuron

A two input AND function

x2
1.5
x1 x2 X1 ^ X2
(1,1)
0 0 0 1

0 1 0
Separating line
1 0 0 x2=(-w1/w2)*x1+
(-w0/w2)
1 1 1
(0,0) 1 1.5 x1

Two categories of AND inputs

 Recognition of AND Inputs using an MCP Neuron

• The slope and the intercept of the separating straight line

are given as follows
-w1/w2= -1 ….. (i)
-w0/w2= 1.5 ….. (ii)
Choosing w0=-1.5 we get w1=w2=1

0 1.5 Σwixi
1 2
The threshold function
 Recognition of AND Inputs using an MCP Neuron

Responses of the AND neuron for different inputs

x1 x2 w1x1+w2x2 Σwixi + w0 y

0 0 0 -1.5 0
0 1 1 -0.5 0
1 0 1 -0.5 0
1 1 2 0.5 1
Recognition of EXOR Inputs using an MCP Neuron

x2
A two input EXOR function
c0 c1

X1 ^ X2 (0,1) (1,1)
x1 x2
0 0 0
0 1 1
1 0 1
1 1 0 (0,0) (1,0)
x1

Two categories of XOR inputs

 Recognition of EXOR Inputs using an MCP Neuron

• There is no single straight line that is able to

separate C0 from C1
– Clearly one threshold logic neuron cannot do
this job
– This can be proved mathematically
Recognition of EXOR Inputs using an MCP Neuron
• Since the desired central region of 1s in the geometry is
created by two straight lines, a solution to the problem
emerges if we first implement each separating line as the
discriminant function of a MCP neuron and then take the
logical intersection of their outputs

X2

L2

(0,0) 1 X1
L1
Two straight lines are required for proper class separation
The architecture of a neural network that can implement the
EXOR logic
X
2
1
L2
0 X
L
x0 , 1
0 1 X
2
1
L
x1 0 1 2X
, L 1
0
1

X
2 1
x2
0 X Recognition of EXOR
L1 1
,
0
1
inputs using MCP
neurons
 Pattern Recognition as a Problem of Machine
Learning

• The problem of machine learning requires to find

a function, say f, to respond “acceptably” to the
members of a training set
– The function, f, may in general be represented as
f:Rn →R
• Automatic pattern classification can be viewed as
a problem of approximating a finite discrete
valued function, g:Rn →{1,2, …,m} where n
denotes the number of features chosen to represent
patterns in the feature space and m the total
number of pattern classes
 Major Applications of ANNs
• Pattern classification
• Clustering/categorization
• Function approximation
• Prediction/forecasting
• Optimization
• Control
 ANNs versus KBs
• ANNs directly compute with sensory information
about the world for replicating intelligent behaviour
– Computation in ANNs takes place at the level of nodes
– Individual nodes and connections do not represent any
concept
– Combined activities of all the nodes in an ANN with a
particular distributed pattern of nodes and their weighted
interconnections cause an artificially generated intelligent
behavior
• KBs in contrast computes with symbols, used for
making an abstract or conceptual representation of
the world, for replicating intelligent behaviour
– Computation in KBs takes place at a centralized place
called the Inference Engine
 Major Sub fields of AI
• Search
– A problem solving paradigm in AI is viewed as a search
process for the solution in the problem space or a space of
problem states.
– The search requires to probe a number of sequences of
inferences in the problem space, at least one of which will
lead to the solution.
• Knowledge Representation
– It deals with tricks and techniques of encoding knowledge,
expressed with descriptive statements, for storing them into
machine ready data structures.
 Major Sub Fields of AI (contd.)
• Machine Learning
– It deals with algorithms for discovering certain
general truths from one’s experiences with various
instances of the problem
• Non-monotonic Reasoning
– It arranges for retraction of old knowledge that
contradicts the new one and also the conclusions
derived with the old knowledge
• This leads to shrinkage or non monotonic growth of the
KB with inclusion of new knowledge.
 Major Sub Fields of AI (contd.)
• Computer Vision
– It is concerned with the development of tools and
techniques to extract useful information form digitized
scene images
• Speech and Natural Language Processing.
– One major objective of speech processing is to design a
computer system that can accept inputs in speech forms and
there after synthesize and generate responses in the same
form.
– It involves Natural Language Processing in addition to
speech recognition and speech synthesis.