Prologue

This book and my life are animated by two passions.

For twenty-five years I have been passionate about mobile computing. In the high
-tech world of Silicon Valley, I am known for starting two companies, Palm Compu
ting and Handspring, and as the architect of many handheld computers and cell ph
ones such as the PalmPilot and the Treo.
But I have a second passion that predates my interest in computers one I view as
more important. I am crazy about brains. I want to understand how the brain work
s, not just from a philosophical perspective, not just in a general way, but in
a detailed nuts and bolts engineering way. My desire is not only to understand w
hat intelligence is and how the brain works, but how to build machines that work
the same way. I want to build truly intelligent machines.
The question of intelligence is the last great terrestrial frontier of science.
Most big scientific questions involve the very small, the very large, or events
that occurred billions of years ago. But everyone has a brain. You are your brai
n. If you want to understand why you feel the way you do, how you perceive the w
orld, why you make mistakes, how you are able to be creative, why music and art
are inspiring, indeed what it is to be human, then you need to understand the br
ain. In addition, a successful theory of intelligence and brain function will ha
ve large societal benefits, and not just in helping us cure brain-related diseas
es. We will be able to build genuinely intelligent machines, although they won't
be anything like the robots of popular fiction and computer science fantasy. Ra
ther, intelligent machines will arise from a new set of principles about the nat
ure of intelligence. As such, they will help us accelerate our knowledge of the
world, help us explore the universe, and make the world safer. And along the way
, a large industry will be created.
Fortunately, we live at a time when the problem of understanding intelligence ca
n be solved. Our generation has access to a mountain of data about the brain, co
llected over hundreds of years, and the rate at which we are gathering more data
is accelerating. The United States alone has thousands of neuroscientists. Yet
we have no productive theories about what intelligence is or how the brain works
as a whole. Most neurobiologists don't think much about overall theories of the
brain because they're engrossed in doing experiments to collect more data about
the brain's many subsystems. And although legions of computer programmers have
tried to make computers intelligent, they have failed. I believe they will conti
nue to fail as long as they keep ignoring the differences between computers and
brains.
What then is intelligence such that brains have it but computers don't? Why can
a six-year-old hop gracefully from rock to rock in a streambed while the most
advanced robots of our time are lumbering zombies? Why are three-year-olds alrea
dy well on their way to mastering language while computers can't, despite half a
century of programmers' best efforts? Why can you tell a cat from a dog in a fr
action of a second while a supercomputer cannot make the distinction at all? The
se are great mysteries waiting for an answer. We have plenty of clues; what we n
eed now are a few critical insights.
You may be wondering why a computer designer is writing a book about brains. Or
put another way, if I love brains why didn't I make a career in brain science or
in artificial intelligence? The answer is I tried to, several times, but I refu
sed to study the problem of intelligence as others have before me. I believe the
best way to solve this problem is to use the detailed biology of the brain as a
constraint and as a guide, yet think about intelligence as a computational prob
lem a position somewhere between biology and computer science. Many biologists te
nd to reject or ignore the idea of thinking of the brain in computational terms,
and computer scientists often don't believe they have anything to learn from bi
ology. Also, the world of science is less accepting of risk than the world of bu
siness. In technology businesses, a person who pursues a new idea with a reasone
d approach can enhance his or her career regardless of whether the particular id
ea turns out to be successful. Many successful entrepreneurs achieved success on
ly after earlier failures. But in academia, a couple of years spent pursuing a n
ew idea that does not work out can permanently ruin a young career. So I pursued
 the two passions in my life simultaneously, believing that success in industry
would help me achieve success in understanding the brain. I needed the financial
resources to pursue the science I wanted, and I needed to learn how to affect c
hange in the world, how to sell new ideas, all of which I hoped to get from work
ing in Silicon Valley.
In August 2002 I started a research center, the Redwood Neuroscience Institute (
RNI), dedicated to brain theory. There are many neuroscience centers in the worl
d, but no others are dedicated to finding an overall theoretical understanding o
f the neocortex the part of the human brain responsible for intelligence. That is
all we study at RNI. In many ways, RNI is like a start-up company. We are pursu
ing a dream that some people think is unattainable, but we are lucky to have a g
reat group of people, and our efforts are starting to bear fruit.
* * *
The agenda for this book is ambitious. It describes a comprehensive theory of ho
w the brain works. It describes what intelligence is and how your brain creates
it. The theory I present is not a completely new one. Many of the individual ide
as you are about to read have existed in some form or another before, but not to
gether in a coherent fashion. This should be expected. It is said that "new idea
s" are often old ideas repackaged and reinterpreted. That certainly applies to t
he theory proposed here, but packaging and interpretation can make a world of di
fference, the difference between a mass of details and a satisfying theory. I ho
pe it strikes
you the way it does many people. A typical reaction I hear is, "It makes sense.
I wouldn't have thought of intelligence this way, but now that you describe it t
o me I can see how it all fits together." With this knowledge most people start
to see themselves a little differently. You start to observe your own behavior s
aying, "I understand what just happened in my head." Hopefully when you have fin
ished this book, you will have new insight into why you think what you think and
why you behave the way you behave. I also hope that some readers will be inspir
ed to focus their careers on building intelligent machines based on the principl
es outlined in these pages.
I often refer to this theory and my approach to studying intelligence as "real i
ntelligence" to distinguish it from "artificial intelligence." AI scientists tri
ed to program computers to act like humans without first answering what intellig
ence is and what it means to understand. They left out the most important part o
f building intelligent machines, the intelligence! "Real intelligence" makes the
point that before we attempt to build intelligent machines, we have to first un
derstand how the brain thinks, and there is nothing artificial about that. Only
then can we ask how we can build intelligent machines.
The book starts with some background on why previous attempts at understanding i
ntelligence and building intelligent machines have failed. I then introduce and
develop the core idea of the theory, what I call the memory-prediction framework
. In chapter 6 I detail how the physical brain implements the memory-prediction
model in other words, how the brain actually works. I then discuss social and oth
er implications of the theory, which for many readers might be the most thought-
provoking section. The book ends with a discussion of intelligent machines how we
can build them and what the future will be like. I hope you find it fascinating
. Here are some of the questions we will cover along the way:
Can computers be intelligent?
For decades, scientists in the field of artificial intelligence have claimed tha
t computers will be intelligent when they are powerful enough. I don't think so,
and I will explain why. Brains and computers do fundamentally different things.
Weren't neural networks supposed to lead to intelligent machines?
Of course the brain is made from a network of neurons, but without first underst
anding what the brain does, simple neural networks will be no more successful at
creating intelligent machines than computer programs have been.
Why has i been so hard to figure out how the brain works?
Most scientists say that because the brain is so complicated, it will take a ver
y long time for us to understand it. I disagree. Complexity is a symptom of conf
usion, not a cause. Instead, I argue we have a few intuitive but incorrect assum
ptions that mislead us. The biggest mistake is the belief that intelligence is d
efined by intelligent behavior.
What is intelligence if it isn't defined by behavior?
The brain uses vast amounts of memory to create a model of the world. Everything
you know and have learned is stored in this model. The brain uses this memory-b
ased model to make continuous predictions of future events. It is the ability to
make predictions about the future that is the crux of intelligence. I will desc
ribe the brain's predictive ability in depth; it is the core idea in the book.
How does the brain work?
The seat of intelligence is the neocortex. Even though it has a great number of
abilities and powerful flexibility, the neocortex is surprisingly regular in its
structural details. The different parts of the neocortex, whether they are resp
onsible for vision, hearing, touch, or language, all work on the same principles
. The key to understanding the neocortex is understanding these common principle
s and, in particular, its hierarchical structure. We will examine the neocortex
in sufficient detail to show how its structure captures the structure of the wor
ld. This discussion will be the most technical part of the book, but interested
nonscientist readers should be able to understand it.
What are the implications of this theory?
This theory of the brain can help explain many things, such as how we are creati
ve, why we feel conscious, why we exhibit prejudice, how we learn, and why "old
dogs" have trouble learning "new tricks." I will discuss a number of these topic
s. Overall, this theory gives us insight into who we are and why we do what we d
o.
Can we build intelligent machines and what will they do?
Yes. We can and we will. Over the next few decades, I see the capabilities of su
ch machines evolving rapidly and in interesting directions. Some people fear tha
t intelligent machines could be dangerous to humanity, but I argue strongly agai
nst this idea. We are not going to be overrun by robots. It will be far easier t
o build machines that outstrip our abilities in high-level thought such as physi
cs and mathematics than to build anything like the
walking, talking robots we see in popular fiction. I will explore the incredible
directions in which this technology is likely to go.
My goal is to explain this new theory of intelligence and how the brain works in
a way that anybody will be able to understand. A good theory should be easy to
comprehend, not obscured in jargon or convoluted argument. I'll start with a bas
ic framework and then add details as we go. Some will be reasoning just on logic
al grounds; some will involve particular aspects of brain circuitry. Some of the
details of what I propose are certain to be wrong, which is always the case in
any area of science. A fully mature theory will take years to develop, but that
doesn't diminish the power of the core idea.
* * *
When I first became interested in brains many years ago, I went to my local libr
ary to look for a good book that would explain how brains worked. As a teenager
I had become accustomed to being able to find well-written books that explained
almost any topic of interest. There were books on relativity theory, black holes
, magic, and mathematics whatever I was fascinated with at the moment. Yet my sea
rch for a satisfying brain book turned up empty. I came to realize that no one h
ad any idea how the brain actually worked. There weren't even any bad or unprove
n theories; there simply were none. This was unusual. For example, at that time
no one knew how the dinosaurs had died, but there were plenty of theories, all o
f which you could read about. There was nothing like this for brains. At first I
had trouble believing it. It bothered me that we didn't know how this critical
organ worked. While studying what we did know about brains, I came to believe th
at there must be a straightforward explanation. The brain wasn't magic, and it d
idn't seem to me that the answers would even be that complex. The mathematician
Paul Erdös believed that the simplest mathematical proofs already exist in some et
hereal book and a mathematician's job was to find them, to "read the book." In t
he same way, I felt that the explanation of intelligence was "out there." I coul
d taste it. I wanted to read the book.
For the past twenty-five years, I have had a vision of that small, straightforwa
rd book on the brain. It was like a carrot keeping me motivated during those yea
rs. This vision has shaped the book you are holding in your hands right now. I h
ave never liked complexity, in either science or technology. You can see that re
flected in the products I have designed, which are often noted for their ease of
use. The most powerful things are simple. Thus this book proposes a simple and
straightforward theory of intelligence. I hope you enjoy it.