257
*
Towards a new theory of information
Tom Stonier
University of Bradford, UK * *
Received 26 July 1991
Abstract. Information science is badly in need of an informa-
tion theory. The paper discusses both the need, and the possi-
bility of developing such a theory based on the assumption
that information is a basic property of the universe. That is,
like matter and energy, information has physical reality. Any
system which exhibns organisation contains information.
Changes in entropy represent changes in the organisational
states of systems and, as such, quantify changes in the informa-
tion content of such systems. Information, like energy, exists in
many forms. These are interconvertible. Likewise, energy and
information are readily mterconverted: A change of 1 J/°K
equals apprommately 10 23 bits. The paper also considers re-
lated phenomena such as "meaning" and "mtelligence", and
argues that the emergence of machine intelligence in the milieu
of human society presages an evolutionary discontinuity.
Introduction
The most important input into modern produc-
live systems is no longer land, labour, or capital-
it is infortizatioti. When you know enough, you can
greatly reduce the requirements for any of these.
For example, when land becomes scarce (and
therefore expensive), as it does in the centre of
large cities, you can build skyscrapers-a technol-
ogy which became available only about a century
ago. Similarly, when tractors replaced horses, huge
tracts of land were freed which formerly were
dedicated to growing oats or other horse fodder.
Every time an automated machine or a robot
replaces a worker, you are dealing with know-how
replacing labour. Every time a computerised sys-
* This paper presented as the ISI Lecture on 17 Septem-
was
ber 1991. It based on the author’s earlier publications, m
is
particular [4,51 and a tnlogy of books which the author has
wntten or is currency working on. The first, Irrformunnn
und the Internul Structure of the UfIIL’erse. was published by
Spnnger m 1990. The second, Bqiond Injormutron the
Natural Hrsturv of Ititelligetice, is m press [7].
*
Mailing address 838 East Street, Lenox, MA 01240, USA
Journal of Information Science 17 (1991) 257-263
Elsemer
tem allows better stock control so that fewer com-
ponents need be kept on hand, you are reducing
the input of capital required for the operation by
utilising information technology. Likewise, the in-
troduction of computerised fuel injection systems
in automobiles-resulting in significant savings in
petrol and energy consumption-represents just
one of a myriad of instances where an input of
information replaces an input of materials or en-
ergy.
In the importance of information as an
reality,
into human economic activities is nothing
input
new. From its very beginnings, all human eco-
nomic activity has relied heavily on information.
The hunter recognising and tracking a spoor ... the
farmer harvesting seeds to plant next spring ... the
weaver using a hand loom... all these were using
information to carry out their various tasks.
Today, information inputs have become domi-
nant : all economic activities are dominated by
technology. Now, all forms of technology are, in
their turn, becoming dominated by information
technology. No wonder that the bulk of the labour
force has come to make its living by processing
information. Two centuries ago the bulk of the
labour force comprised farm operatives; in the last
century, the bulk consisted of machine operatives;
today, it consists of information operatives [4].
Ours, is now an informatIOn economy. Actually,
it goes far beyond that: ours, is an information
society [5].
In truth, that is also nothing new-it has al-
ways been that way. Since the emergence of hu-
man speech, what has differentiated us from the
rest of the primate stock is the increasing ef-
ficiency with which we, collectively, have been
able to accumulate and transmit information.
Furthermore, one can argue that it was the
efficiency with which a society was able to process
and transmit information-creating a &dquo;collective
intelligence&dquo;-that determined its success. That is,
in the course of human history, advanced cultures
came to dominate or displace more primitive ones
-in each case, the advanced cultures were char-
acterised by socially more efficient information
processing systems-a theme shortly to be ex-
plored in greater detail, if briefly.
258
The growth of the economic and social impor-
tance of information culminated in the emergence
and growth of new information-centred profes-
sions such as librarians, computer scientists, infor-
mation managers, knowledge engineers, etc., and
the natural joining together of such professionals
into organisations such as The Institute of Itiforma-
lion Scienlists.
The question
Embedded in the above discussion is the as-
gumption that we know what information is and
that we fully understand its various manifesta-
tions. This, on some reflection, proves to be incor-
rect. The concept of mformation is as elusive as
that of tune: As G.J. Whitrow [8, p. 11] has
pointed out, &dquo;... time has this peculiar quality
which makes us feel intuitively that we understand
it perfectly so long as we are not asked to explain
what we mean by it&dquo;. The same may be said of
information.
The puzzle is this: Is it not strange that with all
this information-related activity around us, we
still do not know what information is? Consider a
parallel situation: What if mechanical and civil
engineers did not understand the concept of en-
ergy ?
It is the purpose of this paper to move towards
the creation of a new theory of information.
Basic assumptions
Let us begin with the following two assump-
tions [6]:
( 1 ) InformatIOn, like matter and energy, is a basic
property of the Ullluerse.
(2) A ny system which exhibits organisatIOn, con-
turns iiiforniaiion. The more intricately
organised a system is,the more information
has accumulated within that system.
As we shall see, information, like energy, may
exist in a variety of forms. It may be transmitted
from one system to another, and it may undergo a
series of transformations. For example, patterns of
dye molecules on a printed page (words), upon
being read become patterns of neural connections
in the brain, which, in turn, upon being read out
loud become patterns of compressed air molecules
(sounds). If the reading happens to be on a radio
station, the patterns of compressed air molecules
are transformed into electrical pulses by the mi-
crophone and, upon transmission, end up as pat-
terns of pulses of electromagnetic radiation. If a
listener decides to tape the broadcast, then the
information would end up as patterns of mag-
netisation on an audio tape.
In the above example we are dealing with
physical patterns organising matter and energy,
albeit of humanly created and humanly trans-
nutted information. However, information, as
such, has existed since long before the appearance
of human beings. For example, the information
which can be encoded in DNA has been around for
at least a thousand million years. Obviously, RNA
also contains such information. Similarly, self-rep-
licating membranes must contain information, and
one can make a good case that proteins do as well.
However, we need not confine ourselves to organic
substances. Crystallographers have become in-
creasingly conscious of the information contained
by crystal lattices, and one school of thought
believes that the earliest genes, giving rise to the
origin of life, involved complex clay crystals [see
1 ]. Not only crystals, but molecules and atoms,
themselves, exhibit the kind of intricate organisa-
tion which implies that they are rich in informa-
tion [see 3]. If the basic assumptions stated above
are correct, then information arose the moment
the universe began to organise itself, i.e., when,
following the big bang, it differentiated out vari-
ous forms of energy, and then matter.
Information and entropy
Changes in the organisational states of physical
systems show up as changes in entropy. By using
Schrodinger’s modification of Boltzmann’s equa-
tion as a starting point, it becomes possible to
derive the following relationship between entropy
and information [see 6]:
S = k log[ 1,/,1
where S =
entropy
k =
Boltzmann’s constant
I =
the information content of the
system
Io =
the information content of the
system when S 0 =

This
relationship between information and ent-
ropy differs profoundly from that of the com-
munications engineers who, based on the work of
Shannon and others, consider information to be
tlirectli, and linearlv related to entropy. Shannon
never claimed to have developed a theory of infor-
mation-rather, a theory of communication. For
mformation scientists, Shannon’s theory is useful
m quantifying syntactlc aspects of communicated
mformation but is of no help in quantifying the
semantrc aspects of a message.
By applying the above equation to the observed
entropy changes which are associated with alter-
ations in the physical states of matter, and by
making one further assumption, it becomes possi-
ble to show that one entropy unit (1 J/°K/mole)
is approximately equivalent to 10 ‘~ bits/mole [6].
This provides the quantitative basis for the
conversion of energy into information, i.e.,
1 J/°K = 102J bits.
Why has information been overlooked?
Every equation in physics contains either a
physical constant, or a reference to combinations
of time and distance (or their derivatives-veloc-
ity and acceleration). Constants define relation-
vhips. Constants are neither matter, nor energy.
They are however, a form of information. The
same argument can be made for time and dis-
tance. Neither represent matter, or energy. Both
measurements of time and of space refer to some
frame of reference-hence they too, constitute a
form of information.
The question, then, arises: If information is
ever present in the equations describing the physi-
cal aspects of our universe, why has it been con-
sistently overlooked?
The answer has two parts to it: First, because
of the ready interconvertibility of information and
energy, it was always possible to fall back on
interpreting changes in the state of our universe in
terms of energy. Only occasionally did energy
seem to disappear and need some accounting de-
vice such as &dquo;entropy&dquo;, or &dquo;potential energy&dquo;. In
general, however, there was no need to consider
mformation. And therein lies the key to the second
reason why information has been ignored: Until
259
fairly recently, there was no need for a concern.
&dquo;Information&dquo;, as such, did not need to be con-
ceptualised and quantified-it was a non-prob-
lem.
There exists a perfect historical analogy: the
concept of energy. The concept of &dquo;energy&dquo; as
something distinct-with a physical reality of its
own-is a relatively recent discovery. The sys-
tematic exploration of &dquo;forces&dquo; began only a few
hundred years ago, and then only in response to
the need to understand the effectiveness of human
inventions which created such forces-cannons,
and later, steam engines: Galileo was a military
engineer studying the trajectory of cannon balls;
Carnot was interested in defining the theoretical
limits of steam engines. Prior to our conceptuali-
sation of energy as a distinct entity, people were
perfectly content to consider something as hot or
cold, the way a material might be hard or
soft ... That is, these properties were intrinsic to
the system-not something which represented a
common denominator which could manifest itself
across all forms of matter.
We are in a comparable historical situation
today: Until this century there was no need to
invoke information as a separate and abstract
entity. The first to see such a need were the
telephone and radio engineers: Charged with the
job of maximising transmission efficiency, en-
gineers such as R.V.L. Hartley treated information
as an independent, abstract quantity, and by 1928
had provided an equation for its measurement [as
reviewed by 2]. More recently, biologists decoding
DNA and crystallographers have, likewise, felt the
need to start to look at information as an entity in
its own right. However, the main thrust for the
recognition that information has some sort of ab-
stract reality which is badly in need of accurate
quantification and a general theory, comes from
the wide range of information scientists who have
had an entirely new historical experience-
processing information by means of a computer.
Structural vs. kinetic information
If information is as intrinsic a component of
the universe as are matter and energy, then, like
matter and energy, it may exist in a number of
forms. Two major categories are structural and
kmetlc information. Structural information is the
260
information contained by a system; kinetic infor-
mation is information which is being transmitted,
processed, or transformed. For example, when
considering a steam engine, the structural infor-
mation resides in the steam engine itself-its
organisation, the placing of the drive shaft, the
size of the boiler, the nature of the metal, etc.
Incorporated into it is the entire history of rele-
vant technologies-not only the contributions of
Newcomen, Watt, Bolton, and the other scientists
and engineers who helped to improve the design
of steam engines, but also the history of metal-
lurgical technologies, mining technologies, trans-
portation technologies, etc., without which steam
engines could never have come into existence. The
structural information contained by any system is
a function of its previous history.
A steam engine provides an information input
to a non-specific energy input such as heat. If we
heat water merely in an open vessel, the heat
becomes dissipated as steam and all that happens
is that the entropy of the system is increased-all
work has gone to waste. In contrast, a steam
engine provides an information input into the
equation so that the heat is converted into mecha-
nical energy. Its information input is so substan-
tial that the steam engine acts as a form of
Maxwell’s demon: It manages to arrange matters
such that high energy molecules end up on one
side of a piston, and low energy molecules on the
other. This information provided by the steam
engine constitutes kinetic information. Kinetic in-
formation is involved whenever one is dealing
with changes in entropy-an increase in entropy
signifying a loss of information.
The two forms of information, structural and
kinetic, may readily interconvert, one into the
other-whenever work is performed resulting in a
stable reorganisation of a system. This process is
traditionally described as involving changes in
&dquo; potential&dquo; energy. An increase in potential en-
ergy (thermodynamically less probable) signifies
an increase in the information content of the
system. It also signifies a conversion of energy
into information: That is, if I pick up a pencil
from the floor and place it on the table, the work
performed shows up as a reorganisation of the
universe. The classical explanation in physics to
account for the apparent disappearance of the
energy, is to invoke the concept of &dquo;potential&dquo;
energy. The explanation put forward here is that
work was expended to increase the information
content of the system-the apparent loss of en-
ergy is compensated for by a corresponding in-
crease in information.
Forms of information
Like energy, information may appear in many
forms. When we switch on the electric lights in our
house, the history of that light energy includes
several transformations. In most of the world’,,,
power generating stations, the initial energy input
is either heat from burning fossil fuels or from
atomic reactions. This becomes transformed into
mechanical energy by the steam turbines, which.
coupled to generating shafts, convert that energy
to electricity. In our home, the electrical energy
becomes converted further into other forms of
energy-electro-magnetic radiation (light bulbs),
mechanical energy (one of the numerous electric
motors in the house, e.g., refrigerators, washing
machines, clocks, hair dryers, etc.), sound (alarms,
speakers). It may also be converted back into heat
(toasters, ovens, hair dryers, etc.).
In the above examples, we are dealing with
both an advanced scientific understanding and a
mature technology: Ever since Joules, we have had
the insight that various forms of energy are inter-
convertible. Engineers feel comfortable with
calculating the interconversion of one form of
energy into another.
Not so with information-at least, not yet.
Physical information may exist in many forms:
The constants of equations describing the physical
world, time, distance, and directIOn, the informa-
tion properties of particles of matter, the informa-
tion properties of various forms of energy (e.g. the
wavelength and frequency of electro-magnetic
radiation)-all represent forms/ information. We
have already discussed the great divide between
structural and kitietic- information.
In general, we also do not understand the con-
version of energy into information. For example,
one of the electric outputs in our home may
involve an electric clock. Clearly here is a case of
converting energy into information-electrical to
mechanical (the hands of the clock) to a measure
of time. But since we are dealing with the human
measurement of time, we are dealing not only with
physical information, but also with meanmg. At

this point in the history of developing a general
theory of information, we are totally stumped.
Information vs. meaning
We must be very clear about the difference
between itiforniaiion and meamng. Information is
an intrinsic property of various systems (in fact,
the universe itself) which exists irrespective of
whether any human or other forms of intelligence
perceive it or utilise it. Meaning is achieved when
the perceived information can be put into a con-
text. That is, information becomes meaningful only
if it can be analysed, compared and integrated
with other information which already exists within
the perceptor system. A word in a foreign lan-
guage possesses information, but may have no
meaning for the listener if the listener has no prior
knowledge of that language. Such a foreign word
cannot be integrated into the knowledge structures
of the listener unless it is accompanied, or pre-
ceded by further information inputs (such as a
language lesson).
Information vs. intelligence
Intelligence is a property of advanced informa-
tion systems. It is not confined to humans or
higher animals. Instead, it represents a broad
spectrum of phenomena, best defined as &dquo;a prop-
erty of any information-processing system which
is able to analyse its environment, then, on the
basis of that analysis respond in a manner which
enhances its chances of survival or reproduction&dquo;
[7].
By this definition, all biosystems possess some
measure of intelligence. However, the concept
needs to be expanded in two directions: At the
bottom end, proto-mtelligel1ce merges with true
intelligence. At the top end, human global society
represents the epitome of a more generalised phe-
nomenon : collective intelligence. Proto-intelligence
is exhibited by certain crystal systems and various
forms of machine intelligence. Some information
theoreticians, e.g., K. Haefner [3], consider that
atoms represent information processing systems
which &dquo; know&dquo; their identity. That is, an atom
appears to know whether it is a hydrogen atom or
an oxygen atom and will reject particles and other
incursions which threaten its integrity.
261
The present author would draw the line be-
tween proto- and true intelligence on the basis of
whether a system exhibits the capaCity to learn.
Traditionally, &dquo;learning&dquo; is defined as an endur-
ing change of knowledge or behaviour brought
about by experience (including training or study).
On this basis, DNA/RNA-based biosystems are in-
telligent because they are able to learn from their
environment and adapt to it. The random muta-
tions which are incorporated into the DNA (or
RNA) of an organism parallel the random excur-
sions into the blind alleys of a maze which the
brain of a rat remembers. Both systems-the rat
brain guiding rat behaviour, and the DNA guiding
the behaviour of the species-represent analogous
systems: Ineffective variations in the coding of the
DNA fall by the wayside as evolutionary pressures
select against them. In contrast, evolution favours
successful variations. Likewise, ineffective neural
patterns representing the excursions into blind
alleys, decay, while patterns representing success-
ful paths, are strengthened. Both an individual rat
and a whole species, alter Juture responses to their
environment on the basis of having acquired a
store of significant information based on experi-
ences of the past.
&dquo;Collective intelligence&dquo; involves the integra-
tion of intelligent sub-units into a coherent unit
capable of solving problems beyond the reach of
the individual units. The clearest examples are
found among insect societies such as bee hives and
ant colonies. Another example is a network of
computers. In both instances, the &dquo;intelligent&dquo;
subunits are integrated into a collective by mes-
sages passing between units. Also, in both in-
stances, the units are programmed to respond to
certain environmental cues (including messages
from other units) by producing specific messages
which, in turn, cause the recipient units to react in
accordance with some pre-programmed set of re-
sponses. In the case of army ants the messages
transmitted involve mainly an exchange of chem-
ical and body signals, while in a computer net-
work they are electronic. The lrntrtatrons of pro-
grammmg in ants is attested to by the fact that
parasitic insects and other organisms, mimicking
ant signals, can readily fool the ants into feeding
them-even while thev are devounng their larvae.
Keep in mind that the brain of an army ant
consists of only of the order of 10‘ neurons, and
that a single army ant cannot reproduce, and
262
perhaps not even survive as an individual. On the
other hand, the collective intelligence of an army
ant colony comprising several hundred thousand
individuals wreaks havoc and totally dominates its
surroundings.
The same principle applies to human societies:
In general, for any given size, the power of any
human group is a reflection of its cohesiveness.
And the internal cohesiveness of a group is a
function of its communication/ information tech-
nology. For example, one can interpret the
ascendancy of European power following the Mid-
dle Ages as reflecting the enormous impact the
printing press, coupled to a Latin alphabet and an
advanced writing technology, made in creating a
&dquo;trans-European brain&dquo;. Printing, as is true for
any major new form of information technology, is
a meta-technology, that is, a sort of super technol-
ogy which accelerates the development of all other
forms of technology [4].
Today, our information technology includes
vastly expanded collective memories such as
libraries, archives, bureaux of records, museums,
scientific collections, etc., increasingly coupled to
sophisticated information storage and retrieval
systems. In addition we have vastly expanded,
a evolve into new, advanced forms of organisation
global communications technology-postal sys-
tems, telegraph, telephone, radio, television, satel-
lites, fax, e-mail, etc. Finally, our capacity to pro-
cess information, both individually and collec-
tively, is increasing by leaps and bounds. These
advances occur because of the rapid strides being
made in the advances of information technology
-strides which involve the interplay between in-
dividual human intelligences integrated into a
global network of endeavours. On this basis it
would be true to say that the collective intelligence
of human global society represents the most ad-
vanced form of intelligence known to us.
The future of machine intelligence
Once it is understood that &dquo;intelligence&dquo; com-
prises a spectrum of phenomena, with &dquo;proto-in-
telligence&dquo; at one extreme, and &dquo;human global
intelligence&dquo; at the other, it becomes clear that
various forms of &dquo;machine intelligence&dquo; do, in-
deed, comprise forms of intelligence. Furthermore,
as we pour our collective wisdom and experience
into the design of ever more powerful computers,
machine intelligence will start to outpace human
individual intelligence sometime during the 21st
century. A primitive, and somewhat superficial
early test of this process will occur during the
1990s, when chess-playing computers, containing
the collective intelligence of a myriad of scientists
and engineers, will beat the world champion grand
masters.
More profound, however, will be the impact of
combining human individual, human collective.
and machine intelligence into new systems of (col-
lective) intelligence. The result will lead to an
evolutionary discontinuity. A few thousand mil-
lion years ago, there appeared exotic self-duplicat-
ing molecules and systems within an organic soup
of life-sustaining molecules. These self-duplicating
systems presaged the emergence of a new, highly
complex, but integrated form of organisation
which we subsequently came to call life. It is
probably not an overstatement to say that the
origin of machine intelligence within the milieu of
human society presages as profound a develop-
ment as the origin of life. That is, the emerging.
ever more highly complex but integrated combina-
tions of human and machine intelligence will
which might best be described as pure intelligence.
Intelligence would be as different from life from
which it would derive, as life proved different
from the complex of matter, enery, and informa-
tion, from which it emerged.
What would a general theory of information look
like?
A good general theoryof information should be
able todefine terms like information, intelligence,
meaning, and understanding with precision and
clarity. It should make explicable these phenom-
ena and allow their quantification. Furthermore,
in the author’s view, such a theory must cover a
wide spectrum of phenomena-with an under-
standing of the physical basis of information at
one end, and a genuine insight into the most
advanced forms of intelligence, at the other.
The above discussion indicates the main out-
lines of these more general, theoretical require-
ments. However, in the long run, following Kant’s
dictum that there is nothing so practical as a good
theory, a general theory of information must be
 263

able to answer specific, practical questions such as without a thorough understanding of the concept
the following: of energy, its theory, and its various applications.
We also live in the computer age. However, the
,4. Structural informatIOn practice of information processing is still largely a
matter of trial and error-much more an art than
(1) How much information is contained within a a science. As for a general theory of information,

simple physical structure such as an atom or a it simply does not exist.

crystal? On the other hand, it was the development of
(2) How much information is contained by a new technologies such as cannons and steam en-
molecule of DNA? gines which required new ways of looking at the
(3) How much information is contained within a universe. It was these intellectual requirements
steam engine? which laid the foundation of modern physics as
(4) How much information is contained by a first practised by Galileo, and of thermodynamics.
computer program, or in a book? Our current experience with computers and re-
lated technologies is forcing us to do likewise.
B. Kitietic information Surely a general theory of information as out-
lined above must have an impact on our under-
(5) How much information is supplied by a physi- standing of they universe. Introducing rniformation
cal system engaged in work. as a parameter into the analyses of physical phe-

(6) How much information is supplied by a mole- nomena-equal m stature to matter and energy,
cule of DNA when it is processed by a cell? and readily convertible with them-must open up
(7) How much information is supplied by a steam the possibility of new insights. If our physical
engine when it is transducing heat into mecha- science colleagues stand to gain from a physical
nical energy? theory of information; we, mformation scientists,
(8) How much information is provided by a com- as we develop acomplete General theory of infor-
puter program or a book, to a person mation, will be able to achieve feats equivalent to
processing the program or reading the book? landing men on the moon.

It is hoped that the above explorations con-

The above aresome-without excluding a stitute a useful beginning.

myriad of others-of the basic questions which
must be answerable. At the moment, with the

exception of question 5 (where it is possible to

measure changes in the information status of a
References
physical system by means of changes in entropy),
we seem to have no satisfactory answers. It may
be helpful to be able to quantify the number of [1] A G Cairns-Smith and H Hartman, Clav Minerals and the

bits required to describe a computer program, a Origin of Life (Cambridge University Press, 1986)
[2] C Cherry. On Human Communication, 3rd ed (MIT, Cam-
book, or a molecule of DNA. But it is not enough. bridge, MA, 1978)
A general theory of information must include a [3] K. Haefner, Evolution of information processing systems,
capability of measuring not only the number of (Project,Evolution of Information Processing, University
bits contained by a system, but also the mealllll[{, of Bremen, Germany, 1991)
[4] T Stomer, The Wealth of Information (Thames/Methuen,
i.e., the oalue of those bits to the system processing
London, 1983)
the information-the programmer, the reader, or T Stonier, The Communicative Society a New Era in
[5]
the cell. Human History (Gold Paper No 5, Foundation for Public
Rotations Research and Education, London. 1985)
[6] T Stomer, Information and the Internal Structure of the
Universe (Springer, London. 1990)
Conclusion
[7] T Stomer, Bevond Information the Natural History of
Intelligence (Springer, London, in press)
We live in the space age. It is hard to imagine a [8] G J Whitrow, The Nature of Time (Penguin, Harmonds-
space programme ever getting off the ground worth, 1975)