Chapter 1.

Introduction to computer
models and neurocomputation
 Artificial Neural Network (ANN)

It is based on the formal and Artificial Set of nature-inspired

statistical analysis of human Intelligence computational methodologies
behavior in the face of and approaches to address
different problems complex real-world problems
Symbolic- Deductive Computational
Intelligence intelligence

• Decision tree • Fuzzy logic

• Expert system • Evolutionary computation
• Bayesian network • Neural networks
• Support vector machine
 Artificial Neural Network (ANN)

It is based on the formal and Artificial Set of nature-inspired

statistical analysis of human Intelligence computational methodologies
behavior in the face of and approaches to address
different problems complex real-world problems
Symbolic- Deductive Computational
Intelligence intelligence

• Decision tree • Fuzzy logic

• Expert system • Evolutionary computation
• Bayesian network • Neural networks
• Support vector machine
ANN vs Traditional AI

Traditional Artificial Artificial neural

intelligence network
Enfoque descendente Enfoque ascendente
Basado en la psicología Basado en la biología
Qué hace el cerebro Cómo lo hace el cerebro
Generalización a partir de
Reglas Si/Entonces
ejemplos
Sistemas programados Sistemas entrenados
Lógica, conceptos, reglas Reconocimiento de patrones
Arquitecturas paralelas,
Arquitectura von Neumann distribuidas, adaptativas,
autoorganización
The brain
The brain can be seen as a complex, non-linear and
parallel computer that processes a large amount of
information. Its ability to organize neurons to perform
certain computations, such as the recognition of objects,
in less than 0.2 seconds, is a calculation speed much
higher than the current computer more powerful.

The brain is a very complex neural network:

• 1012 Neurons (1 billion)
• 1016 Synapsis (10.000 synapsis per neuron)
 The neuron
• A neuron is an electrically excitable cell
that receives, processes, and transmits
information through electrical and
chemical signals.
• These signals between neurons occur via
specialized connections called synapses.
• A typical neuron consists of a cell body
(soma), dendrites, and an axon.
• The cell body or soma processes the arrival
information and determines the activation
state of the neuron.
• The dendrites that emanate from the soma
receive impulses from other neurons.
• The axon transmits the soma impulses to
other neurons.
The neuron as a computer

Artificial neural networks (ANNs) are computing

systems inspired by the biological neural
networks that constitute animal brains.

Simplification

Modeling
Artificial neural network vs Computer
Artificial neural network Computer

• Many processors • 1 CPU

• Distributed computing • Serial computing

• Learning • Programming

• Problems without a known • Mathematical calculation, Problems

algorithmic solution in which the algorithm is known
(We have problem data)
 Artificial Neural Network (ANN)

Massive information processing system composed of simple process units, called

neurons, which is used to store knowledge through a learning process.

Based on the idea of how the brain works in the following two senses:
1. The knowledge of the network is acquired through a learning process.
2. Knowledge is stored in the connections between neurons, called synaptic weights.
Artificial Neural Network (ANN)

Output

Inputs
 Neuron model of McCulloch-Pitts

෍ Output

Inputs
Transfer functions:
1. A set of connections transmits
• Threshold Function
information from other neurons
𝟏 𝒊𝒇 𝒙 ≥ 𝟎
(Inputs). 𝒇 𝒙 =ቊ
𝟎 𝒊𝒇 𝒙 < 𝟎
2. The processing unit adds the inputs
and performs a non-linear transfer
• Sign function
function. (eg Threshold Function).
𝟏 𝒊𝒇 𝒙 ≥ 𝟎
3. The activation value (Output) is 𝒇 𝒙 =ቊ
−𝟏 𝒊𝒇 𝒙 < 𝟎
sent to other neurons.
 Binary neuron model (perceptron)
𝑥1 𝑤1
𝑥2 𝑤2
𝑤3
𝑥3 ෍ f(h) Output
𝑤𝑛
𝑥𝑛
Inputs • Weighted addition of inputs
• Non-lineal function
Transfer functions: • Binary output

𝟏 𝒊𝒇 𝒉 ≥ 𝜽
𝒇 𝒉 =ቊ 𝒉 = 𝒙𝟏 ∙ 𝒘𝟏 + 𝒙𝟐 ∙ 𝒘𝟐 + ⋯ + 𝒙𝒏 ∙ 𝒘𝒏
𝟎 𝒊𝒇 𝒉 < 𝜽
θ: threshold
h: synaptic potential
w: synaptic weights
ANN for AND?
X1 X2 AND
𝑥1 𝑤1
Output 0 0 0
෍ f(h) 0 1 0
𝑤2
𝑥2 1 0 0
Inputs 1 1 1

𝑥1

𝑥2
ANN for AND?
X1 X2 AND
𝑥1 𝑤1
Output 0 0 0
෍ f(h) 0 1 0
𝑤2
𝑥2 1 0 0
Inputs 1 1 1

• W1=1
• W2=1
• θ=1.5 𝑥1

𝑥2
ANN for AND?
X1 X2 AND
𝑥1 𝑤1
Output 0 0 0
෍ f(h) 0 1 0
𝑤2
𝑥2 1 0 0
Inputs 1 1 1

• W1=1
• W2=1
• θ=1.5 𝑥1

𝑥2
ANN for OR?
X1 X2 AND
𝑥1 𝑤1
Output 0 0 0
෍ f(h) 0 1 1
𝑤2
𝑥2 1 0 1
Inputs 1 1 1

𝑥1

𝑥2
ANN for XOR?
X1 X2 AND
𝑥1 𝑤1
Output 0 0 0
෍ f(h) 0 1 1
𝑤2
𝑥2 1 0 1
Inputs 1 1 0

𝑥1

𝑥2
 Neurons with continuous output
𝑥1 𝑤1
𝑥2 𝑤2
𝑤3
𝑥3 ෍ f(h) Output
𝑤𝑛 Continuous output
between 0 an 1
𝑥𝑛
Inputs

Transfer functions:
Sigmoid

1
𝑓 ℎ =ቊ
1 + 𝑒 −𝛽ℎ
Sigmoid function

1
𝑓 ℎ =ቊ
1 + 𝑒 −𝛽ℎ

If the β factor increases,

the function increasingly
resembles a threshold
function with θ = 0
 Neurons with continuous output
𝑥1 𝑤1
𝑥2 𝑤2
𝑤3
𝑥3 ෍ f(h) Output
𝑤𝑛 Continuous output
between -1 an 1
𝑥𝑛
Inputs

Transfer functions:

𝑓 ℎ = tanh(𝛽ℎ)
Types of Neural Networks

According to the Architecture and the Learning paradigm

1. What kind of information is it processed and how does this

information influence? → Architecture: connections and type
of neuron (transfer function)

2. How are the synaptic weights modified? → Learning

Algorithms and Learning Paradigms.
 Architecture

Feedforward neural network Recurrent neural network

Hidden
Input layer
layer 𝑤𝑗𝑖1
Output
layer

Inputs Output

The information flows in one direction Bidirectional connectivity

 Architecture

𝑤𝑗𝑖1

Inputs

Output Multi-layer

2
𝑤𝑗𝑖1 𝑤𝑘𝑗
3
𝑤𝑙𝑘

Single-layer Inputs
Outpus
Learning paradigms

Minimize the mean square error between the

Supervised
output of the network and the desired

Find the most appropriate behavior according

Learning Unsupervised to the structure of the input data.

Based on a trial and error process that seeks to

Reinforcement maximize the expected value of the known
criterion function
Learning paradigms

Minimize the mean square error between the

Supervised
output of the network and the desired

Binary output: Classification

Continuous output: Regression

Find the most appropriate behavior according

Learning Unsupervised to the structure of the input data.

Based on a trial and error process that seeks to

Reinforcement maximize the expected value of the known
criterion function
 Applications of ANN
• Financial market
 Applications of ANN
• Financial market
• Cancer prediction
 Applications of ANN
• Financial market
• Cancer prediction
• Digital image processing
 Applications of ANN
• Financial market
• Cancer prediction
• Digital image processing
• Robotics
 Applications of ANN
• Financial market
• Cancer prediction
• Digital image processing
• Robotics
• Game theory
 Applications of ANN
• Financial market
• Cancer prediction
• Digital image processing
• Robotics
• Game theory
• Energy sector