CITS7212 Computational Intelligence

Introduction

What is Computational Intelligence?

Pseudonyms:
Computational Intelligence
Natural Computation
Nature-Inspired Computing
Soft Computing
Adaptive Systems

A potted history of computing

First there was the Switch

and something called a Computer

http://www.lib.jmu.edu/special/jmuphotopages/stbu.aspx

Then there was the Thermionic Valve

(an automated or controlled switch)

and all sorts of things became possible!

Colossus Mark II

Source: http://cleantechnica.com/category/recycling/

Some even dreamed of a Home Computer!

Then came the Transistor (another switch)

http://ees.wikispaces.com/Historia+del+transistor

which got smaller

and smaller, and smaller...

...but it was still just a lot of switches!

The switches could do a lot of cool things...

They loved logic!
very small logic gates (and lots of them!) on a tiny chip
From logic you could make adders, multipliers, decoders, storage devices,...
This meant you could do lots of arithmetic really fast
so we called them computers

 The switches could also be used to control things, like what sums to do
then you could decide what to do next depending on the answer
And the switches could repeat it as many times as you wanted
the switches could keep working while you played golf!
But it was very laborious setting up all that arithmetic up by hand

So along came programming

So we came up with mnemonics which

were clearer to humans, and the switches
decoded them into control signals
we called this assembly language
ADD A, STO B, JMP C,...

blog.wired.com

In theory you could control the switches,

but actually telling them what to do in
binary was a lot of hard work, and you
often made mistakes

 We found we used lots of patterns of these mnemonics over and over, and
specifying every little step was still time-consuming and error-prone
Why not write human-like statements and get the switches to compile them
into mnemonics (and hence into binary) for you?

This abstraction away from switch language, along with bigger and better
storage devices, allowed us to program the switches for a whole heap of stuff
calculations, graphics, data storage and retrieval,
communications & networking, music, video,...

But ultimately they were still just binary switches...

...and we dont live in a binary world!
Not everything is true or false
Not everything is black or white
how many tall people in this room?
who thinks the Dockers are a great football team?
raise your hand when the following bar becomes red...

Where switches run into trouble

Traditional switch programs run into difficulty with
uncertainty, or missing information
fuzzy concepts or categories
noisy/erroneous information
the analogue world
visuo-spatial, temporal change, inexact, adaptive, abstract

Yet humans and other animals do amazingly well

Huge complexity
extraction of important features from masses of information
Abstraction from detail
dealing with uncertainty, inexactness, noise, incompleteness

So how do we get computers to do these things?

Programming from an alternative perspective

Nature as proof of concept
how does nature do it?
what can we learn? copy? mimic?

Natural Computation (as defined on csse.uwa.edu.au)

Nature is a remarkable problem solver.
From multi-resistant super-bugs to ever more inventive humans,
the mechanisms of construction, adaptation, learning, and
co-operative behaviour found in nature have solved a vast array
of problems in very challenging environments.
Natural Computation is the field of computing that draws from
the successes of natural systems to develop new ways of solving
computational problems in the real world.
The sources of inspiration for natural computation systems
come from a number of closely related areas.
Four of the main areas are outlined below.

Evolution designing the basics

Genes and chromosomes contain the recipe for nature's designs.

Evolution - through the immense power of natural selection
acting over geological time - provides a mechanism for reusing
good designs and improving performance.
Evolutionary techniques - such as genetic algorithms, evolutionary
algorithms, evolutionary computation, and differential evolution have been used widely to solve a huge range of problems in design
and optimisation.

Projects we have conducted locally include:

-

Designing (networks of) comminution equipment for the mining industry.

Designing torpedo tracks for the weapons industry.
Evolving tactics for artificial soccer players.
Solving sports scheduling problems.
Solving 2D and 3D cutting & packing problems.
Debugging logical errors in Java programs.
Evolving neural networks (neuro-evolution).
and many others

Development connecting the components

While genes encode the building blocks for an organism,

the way the organism develops depends upon both its
internal and external environments.
Your brain, for example, continues to develop its hardware
until at least your 20s, and there is evidence to suggest that
it retains its plasticity for much longer.
The way that an organism develops using its genetic recipe and the
factors that control it are still not well understood, and of our four
examples this has had the least transfer to practical systems.

Projects we have conducted locally include:

- Developing topographic maps between the retina and optic tectum.

Learning training the software

While an organism is born with and develops many raw capabilities,

it is through interaction with the environment that it learns to use
these capabilities. You have all seen this in your own experience,
from a small child learning not to touch a hot stove, to the many
hours of training that lead to the expertise of a chess grandmaster
or a surgeon.

In the early days of computing it was often stated that computers

could never exhibit intelligence because they could only carry out
tasks that their programmers had anticipated and pre-programmed
solutions for.
The field of machine learning has shown that programs are able to
develop competencies far greater than those of their programmers,
and this is one the most exciting areas for the future.

Projects we have conducted locally include:

- Learning to play games (Poker, Go, Pacman, numerous others).
- Fraud detection.
- Learning to identify intention in natural language dialogue.
- Learning to tag parts of speech using case-based reasoning.
- Applying computational learning theory (CLT) to planner induction.
- Reinforcement learning in high-dimensional state spaces.
- Learning to compose music.
- Reinforcement learning in small mobile robots.

Co-operation the whole exceeds the sum of its parts

The success of many species relies not just on each individual's capabilities,
but rather on the co-operative behaviour or complex social interactions of
groups. This allows problems to be solved that could not be solved by any
single individual. Well-known examples include ants, bees, many predators,
and, of course, humans.
The ideas behind social co-operation have led to algorithms
such as particle swarm optimisation, ant colony optimisation, and
learning classifier systems.

Projects we have conducted locally include:

- Using PSO and LCSs to derive complex game strategies.
- Using PSO to optimise weights in recurrent neural networks.
- Investigating the convergence properties of PSO algorithms.
- Using LCSs to evolve sets for classification problems.

So what is Computational Intelligence?

Is CI defined entirely by analogy with the natural world,
or can we be more precise?
The key properties of Computational Intelligence are:
Identifying simple mechanisms to produce good solutions,
rather than complex mechanisms to produce an optimal solution.
Exploiting heuristics and simple rule sets with complex emergent behavior.
Adaptation to the environment: CI processes will incrementally improve
their performance with exposure to an given environment.
Using approximate (fuzzy) measures in evaluating
the outputs of processes.
The key technologies (EAs, NNs, ACO, PSO) all exhibit these qualities,
and fuzzy logic is used to design controllers that are tolerant of the
approximative nature of these methods.

What CI is not: Artificial Intelligence

Classical AI is the endeavor of making a machine appear intelligent.
For example the Turing Test is assessed by the perception of an
external agent. The internal mechanism that mimics intelligence
is not as significant as the perception of intelligence.
Consider a key famous success of AI: Deep Blue beating Kasparov in 1997.
The system that beat Kasparov was highly optimized, with an extensive
knowledge base. There is nothing emergent or fuzzy about Deep Blue.
CI is a specific subset of AI,
but while AI focuses on the outcome,
CI focuses on the mechanism.

What CI is not: Cognitive Science

Cognitive science is the study of how intelligence is actually manifested,
rather than simply the perception of intelligence. It combines neuroscience,
psychology, linguistics, philosophy, and anthropology, as well as AI and CI,
to suggest how the human mind actually works.
While cognitive science is also in the intersection of biological models and AI,
its focus is on how the mind works, rather than how we might exploit these
models to solve computational problems.
CI often abstracts the complexities away from cognitive science to find the
simple underlying mechanisms that might be exploited.

What CI is not: Machine Learning

Machine learning is the study of mechanisms that allow a machine
to infer complex patterns and behaviors from empirical data.
Machine learning is used extensively in image and speech recognition,
as well as in data-mining applications.
While similar in nature to many CI techniques, the main difference is that the
core representation of knowledge in machine learning is statistical, whereas
the core representation of knowledge in computational intelligence is
approximate in nature (fuzzy).
Examples of machine learning include
Bayesian networks and Markov models.

What CI is not: Ontological Engineering

Ontology is the study of that which exists. In computer science terms,
ontologies are semantic networks that assign meaning to concepts.
Popular AI often delves into complex understandings of concepts (such as
self awareness), and the semantic web intends to devise a markup language
for ontologies to allow agents to infer the meaning of complex data.

CI does not intend to learn ontologies or derive meanings:

more often it seeks to optimize a behavior or function in a given ontology.

Key Technologies
Evolutionary computation
Populations of solutions competing/co-operating to improve over time
Neural networks
Modelling the connectivity of the brain
Fuzzy logic
Modelling with partial truth and probabilistic logic