• "•aopA-ax ;U.formp.

ti::#a Process ing

rapar #93
s-Sareh 31, 1967

THE TASK ENVIRONMENT OF

Mien Newell and Herbert A* Simon

In the last chapter we examined the requirements imposed by the

task on problem^solving systems designed to discover proofs for theorems

in logi© and mathematics. Xn the present chapter, we continue our study

of the "demands of the situation13 in another class of task environments

that has been much studied! game environments» We shall pay particular

attention to the g.ame of ch-ass g on which most of our own work has focus~

sed, occasionally mentioning Arthur Samuels 4 formidable checkers program

to illustrate specific points. We shall be particularly interested in

comparing the game-playing environment with the theorem"proving environ-

ment to see to wha£ extent: the demands of the situation are the sons or

similar for these two kinds of tasks, and to what extent each task makes

its oijn special demands.

The selection of games as task environments is not frivolous«

there are numerous reasons why they are attractive for problem<= solving

research, and why chess is particularly attractive.

I. Selecting a move in chess is generally acknowledged to be a

difficult problem-solving task* Professional players can devote substan-

tially full time to improving and maintaining their skills in it s and

 players seldom reach the peak of their powers with less than ten
years' experieo&Q. Moreover, after 200 years of intensive study and play,
the game has not become barren or exhausted., Innovations, both in specific
tactics and also in strategy and concepts, continue to be introduced into
oaster play.
2o Because of the vast amount of recorded experience--virtually
all serious games between grandmasters in the last century have been pre
served-=-it is relatively easy to evaluate the quality of a chess-playing
program, and to compare it in great detail with the programs of human players
of different strengths, different styles,, and even different periods in the
history of the game* The protocols produced by a chess program can be com
pared with human protocols in the same game positions and the strength of
the program can be determined accurately by pitting it against human players.
3. Because this task has already been used in previous researches,
particularly in the work of A. de Groot with human chess players and the
several chess programs that will be described in this chapter, new research
can build cumulatively on prior results.
4. The "irregularity" of the structure of chess gives the task
a great deal of the flavor of every-day, garden-variety problem solving
that is absent from tasks like proving theorems or solving puzzles. By
"irregularity" we mean the peculiarity of the moves of the several pieces,
the "exceptional" moves like castling, the strong boundary effects of the
edges of the board, and so on. This point will be obvious to chess players,
and perhaps will appear plausible to readers who are not chess players
when they have finished this chapter.
u;:p f 98

Our description of chess°playing programs, like our description

of the Logic Theorist, will have to include some rudiments of the techno­
logy, but we shall not assume that our readers are, or wish to become,
\
chess players. We shall assume that they have some familiarity with a
board game like checkers, chess, tic~tac-toe, or Go, but we trust they
will not be seriously handicapped in understanding the implications of
the analysis for problem solving if some of the specific technical terms
are rnfamiliar to them. Any problem-solving task we might study involves
a technology; we believe that very few of the characteristics of this task
ar'2 idiosyncratic- We think most of them are observable in a wide range
•:: problem^solving (or at least game-playing) situations.
Finally, we remark that the authors (collectively) can be regarded
strong, but not expert, chess players, conversant with the literature
of the game and able to understand the discourse of masters, although not
able to emulate their practice."- We have organized our discussion in seven
section?. The first section compares the task environment for game playing
with t'.ie task environment for theorem proving. The second, third, and fourth

sections review the history of chess programs for computers, comparing

s,'/eral of the programs that have been proposed or constructed to see what
light the comparison throws on alternative approaches to this problem-
solving task. These sections provide additional examples to illustrate
many points in our discussion of selective heuristics in previous chapters.

I/ We have had valuable assistance from a number of strong players, including

Edward Lasker, Adriaan de Groot, and George Baylor. They have been very
helpful in assessing the plausibility of our interpretations, but should
not, of course, be held accountable for our conclusions.
call? $98

The fifth section examines further the respective roles of

search processes and "noticing" processes in game-pleyii\g programs. We

will see, when we come to consider human chess play in later chapters, that
a
differences in ability to notice relevant features of/chess position is
one important basis for differences in chess-playing strength. In the
present chapter, we vill show how the nature of the task environment in­
fluences the relative significance of noticing and search as problem-sol­
ving strategies.
In the sixth section, we will consider a phenomenon that is
peculiar to competitive situations like war and chess--the notioi of the
"initiative." We will show how holding the initiative reduces the amount
of exploration required to find a good move, and will illustrate the point
by exhibiting a simple, but powerful, program for finding checkmates. In

the seventh, and final, section, we will try to draw together the thread
of discussion.

Gaaass Compared With Theorem Proving

In games like chess, checkers, and tic~tac=toe, the players mek*>

alternate moves. The rules of the game define for a player a finite set
°f jggajL moves from which he must make his choice when it is his tu:tn 0
Making a move creates a new situation in which a set of legal moves is now
defined for the opponent. From the starting position of the game, we can
generate the legal moves, then the legal replies for each of these legal
moves, then the legal replies to each of those replies, and so oh. In this

way, we obtain a branching tree (of enormous size for interesting games)
C1F $98

2/
that describes the set of all possible plays of the game. -

The game tree is analogous in important respects to proof

trees in theorem proving. Consider a formal system consisting of sev­
eral axioms aud a single rule of inference. We can regard the set of
axioms as the "root" of a proof tree. Each new proposition that can be
obtained from the axioms by one application of the rule of inference is
a brarch from the root. From each of these branches, new branches can be
obtained, and so on. (Strictly speaking, the structure will bs a tree
;r.ily if the rule of inference employs a single^ premise"-modus ponsns employs
two. Otherwise, each new twig may have more than one ancestral branch.) To
choose a move in a game, one explores the game tree to learn the conse~
quences of different moves; to discover a proof for a theorem, one explores
the proof tree to learn what path leads to the theorem.
Although the analogy is useful, it is not exact. First, we have
already noticed that if a rule of inference uses more than one premise, the
''•proof tree" is not a tree but a lattice°like structure.
Second, in the proof tree, or lattice, all moves are made by one
player, while in the game tree, alternate moves are made by opposing players
A proof is demonstrated by exhibiting a single path that leads to the
desired theorem. A winning move in chess is demonstrated by exhibiting a
subtree of the game tree having a so-called minlmax propertys that for each
move of the opponent at a given branch, there exists a move for the player
always leading, through a succession of such branches, to a winning position„
(See the next section for detail of this concept.) A proof is a definite

2J The concept of game tree was apparently first formalized by Zermelo in

1912. The concept was used informally in books on chess and checkers
severelt|hundred years earlier. On the history of the concept, see
Denes Konig, Theorie der Endlichen and Unendlichen Graphen. (Leipzig:
Akademische Verlogsesellschaft, 19S6), Chapter 8.
plan, whilfc a "win" is a strategy setting forth a tree of paths for all
possible moves of the opponent «> This distinction has important conse
queries for the nature of game-play ing heuristics, as we shall see
Third, we must not confuse the problem-solving search tree,
Taen there is one, with either the proof tree of theorems, or the game
tree of chess* The subproblem tree generated by the Logic Theorist, for
example, while it is working backwards, is not a t?ee of theorems, but a
tree of propositions from which the theorem to be proved can be derived
(with the help of axioms and previously-proved theorems). In most extant

game-playing programs, the problem-solving activity consists specifically

o£ a search out along some or all of the branches of the game tree; but
a
this is not an essential characteristic of/game-playing program. Indeed,
we will describe some game-playing schemes that operate in spaces differ­
ent from, and larger than, the tree of legal moves.
A fourth important difference between theorem-proving and g«mes
derives from a difference in irreversibility of actions. In theorem-prov­

ing, the aim of the problem-solving activity i§ to find the path that
constitutes the proof. If the search actually takes place in a space of
proofs (or potential proofs, as in the Logic Theorist), the task is to fiv.d
a whole path, but there is no penalty for going down blind alleys during
the search other than the cost of the search time that is involved.
In game playing, the aim of the problem-tiolving activity in any
given instance is to find a move-°i.e.. a first step on a path through
the game tree. Once that move has been selected and aade, it cannot be
undone if it later proves to have been wrong. On the other hand, although
the player may have explored a whole tree in evaluating the move he finally
GIF #98 °7-

cbose, ba is not committed to following any particular path on that tree

(or even sticking to the limits of ths tree if it is smaller than the
legal move tree) beyond his first move. After the opponent has replied,

he cm reconsi'ier his whole future path.

Henie, in game playing, the first move in the game tree from
thu current position occupies' a completely different role from subsequent
mcves. Tha latter are explored not with the aim of selecting a path or
strategy /ree, but solely for the information they may contribute to the
evaluate .*A of the initial move (i.e., the move to be made in the current
pesitici). From this standpoint, proof lattices are of the "essence" of
theorer-proving tasks, while paths in the game tree are only incidental,
although they may have great informational value, to the task of choosing
moves* It is impossible to conceive of a theorem prover that does not
discv/er proofs; a game player that does ot search in the game tree is wholly
conceivable,
A fifth difference between proving theorems and playing games is
that the task in the former is to find a, proof—any one will do--while the
task in the latter is to find the best move, or, if there is no way of
guaranteeing this, at least a very good move. The suitability of a move

can be evaluated only in relation to an assessment of what other moves

nay be available. The one exception is the case where a move can be shown
to lead definitely to a forced win (i.e., a win no matter what the opponent
does). Here, as in the theorem-proving case, the problem solver need search
no further unless esthetic criteria lead him to prefer one winning strategy
r one proof, respectively) to another. We will mention such esthetic
considerations again later in the chapter.

Computer Chess-Playing Programs

A convenient way to gain insight into the nature of the chess

task environment is to examine in historical order some of the attempts
that have been made to construct chess=playing programs for computers,
Because the total population of such programs is becoming sizeable (we
are familiar with at least eight that have been realized on computers,
exclusive of programs specialized for mating combinations), we cannot dis­
cuss them all. Instead, we will select for discussion some that have
been of historical importance in the development of the field, or that
illustrate alternative approaches.
a
Since the idea of/chess-playing automaton (as well as some fraud­
ulent realisations of that idea) is much older than the electronic digital
computer, it is not surprising that proposals for chess-playing programs
were put £01ward almost simultaneously with the invention of the modern
3/ We shall consider here two early proposals, those of Shannon
computer. ~~
and Turing, as well as several programs that have actually been run on
machines «

Shannon's Proposal^

The relevant history begins with a paper by Claude Shannon in

1949 ( }. He did not present a particular chess program, but discussed
many of the basic issues involved in constructing one. The framework he

introduced has guided much of the subsequent analysis of the problem.

3/ Edward Lasker, in The Adventure of Chess, devotes chapters 10 and 11

to the pre-coraputer automata and the electronic chess player, respectively,
GIF #91

The relation of moves to features is a relation of means to

ends. The feature defines a possibility for gain (e.g., an enemy piece
under attack) ©r for loss (e.g., one s s own piece under attack). Moves

are possible means, to be evaluated? for achieving the gain (e.g., cap-
ture the piece) or averting the loss (e.g., interpose between attacker
and attacked). Thus 9 NSS has a built-in knowledge of Important possible
means -end relations. This knowledge enables the program to build a
bridge directly from a present situation to possible actions for trans­
forming it in desired directions; instead of being required to consider
large numbers ©f actions, systematically or at random, with the hope
that some may turn out to be relevant to the desired goals in the given
situation. In later chapters we will encounter numerous examples of

similar means-end structures, in computer programs and in human problem-

solving behavior. Meane«end analysis „ we shall seej, is one of the cen

tral tools for exploring complex task environments efficiently,

The NSS program also stores chess knowledge in the organi gag i on
of its means -end processes s specifically, in the sequential order in
which goalg; are considered, the order in which features are detected
the order In iihich relevant moves are generated. Material balance is
considered bcfrre center control, and center control L
Moves of center pawns are considered, as means toward center control „ be
fore preventitive moves or preparatory moves.
In chess, as in real life, the amount of action that can ba ac
complished in e given time (in one move, in chess) is limited. In order

to act effectively, it is not important, or even relevant, to cons-ids*-

all the things that "might be nice to do." If one's Queen is attacked,
developing one*s Rooks becomes a secondary consideration, which is not
even worth attending to unless it can be combined with the more impor~
tant task of saving the Queen. This is the principal reason why the.

HSS program's flrst-acceptable-move rule produces sensible movas aa

often as it doe^s (1) the move is always proposed for a raason<=-=>it Is

relevant to a goal in the light of specific features of the position|

(2) the' most pressing problems and important considerations are taken up
first; (3) since a single move exhausts the program's capacity for
action, there ie no point in dealing with the less important features 0
The NSS- program illustrates a particular approach to the de-
\

sign of systems for coping with complex task environments. The program?

is aimed not only at making good moves, but at doing so for the right
reasons—that is, in the light of consequences deduced from noticing
certain circumstances in the situation, 'in a task environment as coin=

plex as chess, part of the success of play—of fitting behavior to. the
objective requirements of the situatlon«~depends on the emergence of ap­
propriate concepts. Without them, it will be impossible for a system,
even a very fast one, to explore sufficiently to discover strong moves
and to make reliable evaluations. Chess history can be written largely

in terms of the gradual emergence of a few central concepts: piece values,,

mobility, development, center control, and so on. One should not expact

the equivalent of such a concept simply to emerge from computation in

specific positions, where the computation is based on quite other features
of the position.
SIP i 93 -60-

NEWER CHESS

All the programs we have discussed were in operation by

1960 , A number of programs constructed more recently appear to b<g

distinctly stronger players than those we have examined. We are not

familiar with the detail of construction of all of them, but we shell

mention two that are close to Bernstein's program in general organ"
ization and approach. The improvement in the quality of their play is

due mainly to their incorporation of better chess heuristics, rather

than to novel organizational principles.
At least two relatively strong programs have been constructed

in the United States-«by Kotok et aK, and by Greenblatt ejt aJL^, and at
least one in the USSR. A published description is available only for
the Kotok program, but we are familiar with some of the structure of the
Greenblatt program from personal communication.

The Kotok Program

The Kotok program generates all legal moves and evaluates

them provisionally; it then selects the four most promising for analy­
sis « Analysis is carried eight plies deep, the four most plausible con­
tinuations being considered for each position. If the resulting position

is not dead, pending exchanges are carried cute

Q
To avoid evaluating 4 «65,536 positions at each move, the

program incorporates a so-called alpha-beta heuristic that lops off »m

promising branches. The essential idee of this heuristic is as follows;

Suppose that the value of one of the four replies to a certain move is
already known to be £ (because that reply has been evaluated). But, by

the minimax principle, the value of the move is the minimum of the values
JiF £98

WHITE BIACK WHITE BUCK

(Kotok) (Expert) (Kotok) (Expert)
f

14 PxP N-B3
1 P~K4 P-K4
2 N-KB3 N-QB3 15 P~N7! NxQ
16 PxR^Qch B°B1
3 B~N5 P~S1
0-0 P-SCB4 17 NxPch K-Q2
4
PxB 18 NxR B-K2
5 ByJNch
R~£l H-XB3 19 QxP KxN
6
B-K2 20 RxB N-B3
7 N-QB3
P-Q4 B»K3 21 Q-B7 K-B1
8
22 RxPch K^Nl
9 Q)?xP Q?xP
B-Q4 23 R=K7ch K-R1
10 PxP
24 R=R7ch K-M1
11 NxB Nx25
12 IfeP S.-Q1? 25 Q»N7 laate

13 Q-RSch! P»N3

Remove Black's Queen. The Ruy Lopez opening was agreed upon.

The Kotok Program v. an Expert

At odds of a Queen

(1962)
Table VIII
of the fear replies considered 5 hence the value of the move cannot be
greater thai* cv<, Suppose that oae of the alternatives to this move has
already been evaluated, end its value exceeds v;. Then there is no need

to explore further the other replies to the move in question,, for the
move is known to be inferior to ona of its alternatives „ The alpha-beta

heuristic can reduce drastically the mamber of moves considered „

For each terminal position generated by the program, a auiner=*
ical evaluation function takes into account measures of material value 9
mtbility, development, center controls, checks, pins, and pawn structure.
The moves are evaluated by a minisaax procedure from the terminal positions
In Table „ we give the moves of a game played by the Kotok pro
graw. (White) against en Expert (Black) who g&ve the program odds of a
Queen^-mnd losto Taking immediate advantage of a mistake in Black 8 s 12th
move, the program psesses on to a win. Previous to its attack, the pro
f
gram had developed soundly, exchanging pieces whenever possible (which it
should do, since it is ahead in material) . The other games played by

the Kotok progress though interesting., are not nearly ar. spectacular as

this one. A variant of the program is now engaged ia a ,feur°g£me cor­

respondence match with the USSR program 3 but no games have been completed
yeto

the

A program constructed by Richard Greenblatt and his associates

in recent months gives us our first really accurate information about play­
ing strength in relation to huraan players. Since the program is under con­

tinual revision, its playing strength does not remain constant; o*j.r ccinmants
 #98

WHITE
(P~1505)

P-K4 P«QB4
P~Q4 PxP
3 QxP N»QB3
4 Q-Q3 N»B3
5 N-QB3 P-KN3
6 N»B3 P»Q3
7 B-B4 P-K4
8 B~N3 P-QR3
9 o°o~o P-QN4
10 P-QR4 B-R3ch
11 K-N1 P-N5 11
12 QxQP B-Q2
13 B-R4 B-N2
14 N»Q5 Had? 1
15
16 QxQ N~B4
17 Q-Q6 B-KB1
18 Q-Q5 R-B1
19 HxP B-K3
20 Q2d?chi: R3CQ
21 R-Q8 rsate

Greenblatt Program v. Glass C Player

Table IX
refer to its performance in the Winter of 1966-67 0 The program eoai^

peted in two totrnaments with rated players. In the fi'cst tournsegnt

it drew with a player holding e 1365 rating Class D), and lost four
other-games, St. was provisionally given a rating of 1239 (Class D) f

In the second tournament t it defeated a player with a 1505 rating

(Glass C), and lost four other games., Aside from a serious blindness

about its or>m ^-;.ng safety, its pl&y may evsn be of Cless 3 caliber.
The steiore' of the program's tournament gasss with." the Class C

player < it defeated is shown in Figurs <, In the first nine moves s

the program concerns itself systematically and vigorously with center

control and devtilopment„ (White's third move loses a tempo, but can

hardly be viewed as "amateurish, 11 since the first three moves of both

players are identical with a game between Hisses and Tarraach at Berlir*

in 1920.) Black meanwhile loses time with premature pswn moves, and by

the ninth move, White has a distinct advantage in development 0 Black's

eleventh move is a blunder, instantly exploited by White's surprising

reply On his 14th move, Black falls into a trap that loses his Queen,

The game is now won, bist it is finished by White with « moat elegsnt

checkzsate, beginning with the Queen sacrifice on Move 2,0, and employing

precisely the same motive as Morphy's checkmate of the Duke of Brunswick

and Count Hsouard. It should be observed that White's early success in

this particular game is not unrelated to its ignoring considerations ox

its own JSiiag safety after Black's Move 11.
The program's selection of several book move* is not an isolated

incident. In a gams in which, playing Black, it defeated H. Dreyfus, its

 '63 ^

first four moviJJ were "books'* *- ts human opponent being the first to

depart from acctipted lines. Similarly, the Kotok program, in two

published games in which it played White found book continuations for

three und five noves, respectively, <md in the latter ease th*
th-z-'i departed from Isooko
The program of Greenlylatt is very imaeh like Kotok's,

that it incorporates superior chess heurisfcic,So It considers at least

six continuations at e£ch mcve 9 but only explores four plias deep, mi-
less it has to go further to reach a dead position. The alpha-beta

heuristic is uscid to prune the exploration tree* Typically, the pro-

4
gram ei&siaines for fewer than 6 «1296 posit ions „ In a 2 7-move game £h*at
the program played (and lost) against: an expert 9 the median number of

moves considered was 1,057. The alpha-beta heuristic i>ma used a median

number of 174 times to prune branches. The smallest number of moves con

sidered was 162 (at the first move), and the largest number, 16,613*

The fact that the median was a little over 1,000 shows that the pruning

by the alphas-beta heuristic rather mora than compensated , on average 9

the extra moves generated to reach dasd pasit ions Thiss ? only the

program is more selective in its search than the Greenblatt program

The playing strength of the latter provider further evidence of the ssib

tutabillty of small amounts of chess sophistication for large amounts of

search*
 -if- Jo
^-.Q

STATIC AND DYKAHXC EVAI&ATIGtf

A chess game is won by checkmating the opponent e s King (i 0 t-, 9

placing him in a position where he cannot avoid capture}., Since chess

programs cennot explore ahead exhaustively to terminal positions., they

must use various surrogate procedures to select good moves„ These pro

cedures, all involving the extraction of information from bcerd posi­

tions j, fall intto two laain classes? £ f:stis evaluation functions; applied
to noa~terminal, but relatively "dead/5 positions; and procedures for
detecting features of a position that: may b& used to generate plausible
aoves,
Both classes of procedures can be viewed as j3re<iictlon procedures<

The static evaluators assign to positions numbers, or other descriptors,

that are designed to predict which positions are more promising for
achieving & wit? (or averting a loss). Where the evalu«tors are scalars s

the higher the value s the better the winning chances. (We will not try

to define "chance" in this context, beyond remarking that a probability

interpretation of the term would be strained.) A good evaluation function

is one that orders positions reliably according to the.lr promise„

The plausible move generators can be viewed in the game way<=--

they *ise present features on the board to evoke relevant end desirable
actions. Thus* they predict which moves and continuations will lead to
the highest-valued and lowest-valued positions. There are two steps to

the process: tbe detection of a feature, end the generation of a move

th&t ia relevant to that feature. It may b© detected, for example s that

a piece is attacked; this feature leads to proposing such actions as

capturing the attacker, moving the attacked piece, defending the at-

tacked piece3 or interposing between attacker and attacked„

These procedures build eoimeetionc between two spaces, the

space of "states of the -worid"«»i 0 ® 0 r board positions end the space of

actions that change the world from one space to another« The desired

connections (the chess sophistication incorporated in the generators)

run from partial descriptions of pre&/&t states of the world to actions

that offer promise of leading to & more highly desired world* Much evsry°

day knowledge and skill consists of such connections 2 :,f the bo&rd is

too long, apply a saw to it; if the soup is too hot, blow on it; if the

bread is across the table, don*t reach for it, but ask for it. Thus the

connections are predictive s if 1 apply a saw to SOT*®tiring s it will be­

come shorter; if I blow on a liquid, it will cool; if & ask for some-

thing, it will be passed to me. They express our knowledge of causal

relations that hold in the worldo

What is the nature of the static information available from a

chess position that predicts these causal relations? To what extent is

it idiosyncratic to chess, and to what extent geiser&lirsabla to other sttis-

at!on«j? We can ask. these questions both about static evaluation functions,

and about feature detectors,

Let ES consider firr£ ch© static evaluation -functions ?ieed ii

chess programs* Whatever their di£i;erenc(U» iii dcficf.?., they all have #

monotonous similarity! piece value, mobility, developiaant, center control.

King safety arc*, prominent in all (except for those programs that use only
a subset of th&£<e elements)» Piece values es.n be viewed is* eifher of

two ways.. They can be viewed as statements of equivalence based on chess

experiences that you are about as well off with a Knight as a Bishop 9

'with either of f:hese as with three pawns 9 and so on. On the other hand;,

it can be shown that there is a close correlation betuean piece values

and mobility that the value of each type of piece is very nearly pro-

portional to the average number of squares en the board that it can st»

tack. From this viewpoint, piece values con ba interpreted es estimates,

or predictors, of average long«run mobility? as dietiuguished from measf^e

of current short-run mobility.

Center control is also closely related to mobility, for the

player who controls the center squares can move his pieces from one side

of the board to the other, while restricting his opponent 8 & movements«
Measures of development are measures of short-run mobility„ for developed
pieces are those that have been brought into play from their initial posi­
tions. A measure of Eing safety, on the other hand, combined with a

mobility measure s like relative development, can be used to predict the

likelihood that a successful tcating attack can be mounted<,

We note that mobility, in one form or another, plays a central

rols in almost all of these evaluations. This concept is of course not

peculiar to chess, but is equally important in other gamas as well as in

military strategy and tactics. (Bridge players may wish to try to Inter­

pret the squeeze play in mobility terms, and to note its relation to the

forking move in chess or checkers.) In an early form of the very suc-

cessful checker playing program designed by A. Samuel (Feigenbauci and

pp« 100-103), the five most important terms in the evaluation
CIP #98

function (as well as most of the others), are compounded of measures

of mobility, center control, and denial of occupancy (to the opponent).
Tic-tac>°toe players know that the center square (which lies on
four arrays) is generally the strongest, the corner squares (on three ar­
rays) are the next strongest, and the side squares (on two arrays), the
weakest. In that game there is only a single "trapping" situation which
violates that principle. (A trap ic •& course of action that looks at­
tractive, from the static description, but has undesirable consequences
foreseeable only by dynamic analysis.)
Why does mobility play such a central role in the evaluation
of positions in these games? In a general sense, "high mobility" is
equivalent to flexibility, or "lots of possibilities"—and if there are
lots of possibilities, some of them should lead to good outcomes. But
there is more to the matter than this.
First, some changes in position in chess are reversible, others
are irreversible. Pawn moves (including "Queening" pawns), castling
moves, checkmate, and captures of pieces are irreversible, others are
reversible. Conseqiftently, a change in relative piece .values, through
captures or Queening, has long-term effects. (Subject to many exceptions,

a one-pawn advantage in piece value is enough to win a chess game.)

Second, advantageous irreversible changes in chess are produced by super*
iority of force at specific locations on the /board. Superior force

here means superior numbers of pieces, regardless of value, since each

piece can capture exactly one piece (of any value) in one move. Hence,

short-run advantages in mobility can be used to achieve local superiority

of forcej £°£ purposes of checkmating the opponent s increasing rela­

tive piece value through a favorable capture 8 or gaining control of. an

important squares, The latter two outcomes heve the effect of increasing

middle-run or long-run mobility., which can be transformed, in tiirn,, intc

short^run mobility, and thence into local superiority of fcrea*

The most spectacular kind, of termination of a chess gsme Is

sheeted ting the I'&ing early in the game 9 while most of the pieces are

still on the boa:rd° A player who observes that the opposing King is

somewhat <S2S£CJSsd ; or only lightly protected, and that his o-*n pieces

are better developed (mor-B mobile} than his opponents, may search for

moves that bring about losal superiority ©f force in the izrsnediste aeigfc-

borhooci of the e:*possd King. He sen afford to sacrifice important

pieces '{long-run mobility), if ^.his permits him to achieve the local

superiority of force that will win the game immediately. If the de-

cision is reached in the dhcrt~ruxi, piece values decoios irrelevant . Bat

of course, & sacrificial attack that fails mesns long->rii,n disaster^ henca

the outcome casst be predicted accurately. !"?e will hava more to say about

checkmating attacks in the next section.

Since mobility, in its various forms, is such an important fac­

tor in determining outcomes of chess positions, the features that are

detected for the purpose of activating the plausible move generators must

also relate to mobility. Detecting an underprotected pieca raises the

immediate prospect of a capture that will change relative piece values

of the two sides- Detecting an opening file reveals the prospect of a

Rook move that would increase the mobility of that piece (by allowing

it subsequently l:o move along that file).

 CI? $98 =69-

In games like chess, where each of the two players makes a

single move in turn, forking moves represent an important form of short -
term superiority in mobility. A forking move is a move that attacks
two points simultaneously, securing temporary superiority of force at
both, and threatening immediate, irreversible damage»-i.e., the capture
of a piece or a favorable exchange—at both. More generally, a forking
move nay be termed a double-function move. Double-function moves can
be parried only by double-function replies. Among the important fea­
tures of a chess board that must be noticed by a program are those that
create prospects for double-function moves—e.g., undefended forking
squares.
Two main conclusions may be drawn from this discussion of
static ©valuators and plausible move generators. First, the central

evaluative concepts in chess are not specific to that game, but arise
in other competitive situations, and even in non-competitive ones where
there is uncertainty about the future. Most evaluatora in chess relate
to one or another form of mobility, and while specific measures of mobil°
ity may be peculiar to chess, the general concept is not. Mobility plays
an equally central role in checkers and in warfare. It is closely related
the
to/concept of flexibility that arises in decision makiug under uncertainty.
Second, the discussion of noticing processes suggests a way of
viewing problem solving that is somewhat different from the picture of
search through a large maze. Problem solving, in this alternative view,
involves finding features of a present situation to which experience
(personal or learned) iias associated actions that commonly lead to better
 #98 -70

situations. If a sufficiently rich set of features is known, and suf

ficiently strong causal regularities between them, their associated ac-
tions, and the consequences of the actions, then problem-solving becomes
a matter of noticing rather than a matter of searching.
The history of chess-playing programs has shown a general trend
toward substituting more sophisticated noticing processes for large
amounts of speedy search. The NSS program represents the extreme in this

respect, but Greenblatt's program though calling for considerable search

is far closer in spirit to the NSS program than to the Los Alamos
program. This is not to say that such a powerful set of features could

be defined for chess as to dispense with search altogether. Whether this

not
is possible is not known, certainly it has / bc*o done either in chess
programs or in human chess play. (Exhibitions by grandmasters of suc-

cessful simultaneous play against many (thirty, fifty, or more) weaker

players remind us that a grandmaster's noticing program is more than a
match for a club player's tree search.)
There is a part of chess the discovery of checkmating combin-
ations- -where outcomes must be determined accurately, and search appears
essential. In the next section, we will discuss this particular aspect
of chess. One of its interesting features is that it calls for tree
search resembling that in theorem proving more closely than it resembles
the kinds of tree search we have discussed so far in this chapter.
CXP $98 -71-

CHECKMATING COMBINATIONS

An ironic piece of advice that chess players give to novices

is ° "Always check; it may be Matel" The advice is ironic because many
checking moves are easily refuted and result only in loss of time for
the attacking player.
Nevertheless, series of checking moves that end in mate-
mating combinations--provide much of the spectacular in chess, the
"brilliancies11 comparable to the final smashing cavalry charge of an
(eighteenth century) army that has first moved into position. Some of

the recorded mating combinations are as many as eight or more deep (that
is, sixteen ply), and these give rise to much of the mythology about
the mnemonic and visualizing powers of grandmasters.
We shall describe here a mating combinations program (MATER)
sufficient to discover a great many such combinations, including some
of the most spectacular in chess history. The basic ideas imbedded in
MATER are: (a) to generate only checking moves, and (b) to abandon any
line of analysis as disadvantageous if it allows the opponent more than
a very few (more than four, say) legal replies. Apart from these two
principles, MATER incorporates a few important rules that determine the
order in which the analysis tree will be explored. The significant things
that can be said about this program are: first, that it is remarkably
powerful in finding mating combinations, some involving sacrifices of
Queens and other major pieces; second, that it accomplishes this with a
very modest amount of exploration.
 Mobility, more specifically, restricting the opponent's
mobility, lies at the heart of MATER, First, the condition of check-
mato is itself a matter of mobility. The opponent is checkmated when
(!) his King is under attack; (2) he has no legal move, of King or
other piece, that removes the King from attack. This means, in particu°
lar, that all the squares adjacent to the King—to which he might con­
sider moving—are either blocked by his own pieces or are under enemy
attack. Thus, the checkmated King is under attack and immobile.
Second, mobility is at the basis of the move generator in
MATERo A checking move, attacking the King, must be dealt with at once
by a reply that relieves the attack. Hence, checking limits severely
the moves available to the opponent. HATER generates checking moves,
and all legal replies to each checking move. (The reason for generating
all legal replies will be explained presently.) Those checking moves
that tend to restrict the opponent's mobility most severely—double
checks and discovered checks—are considered first.
Third, mobility is the main criterion for determining along
which branches of the search tree MATER will pursue its search. Of all
the moves that have not yet been evaluated, that one is selected next for
exploration that allows the opponent the fewest replies. A move that
permits more than four replies is rejected out of hand as unlikely to
lead to checkmate.
The organisation of MATER*s search may be described as follows:

1. The checking moves in the starting position are

generated, and all legal replies for each. The checking
moves are put on a "try list." (The try list is the
precise analogue of the subproblem list in the LOGIC
THEORIST o
GIF f98 ~?3=

2. Moves are put on the try list in order of number

of replies (fewest replies first). Among checks with
the same number of replies, double checks and dis­
covered checks are put first.
3. The checking move at the head of the try list is
selected. (Call this move M.) Checking moves and
their replies are generated for each of the replies to
M, and the checking moves placed on the try list, as
in steps 1 and 2. If any replies to M have no checking
moves, M is labelled NCMATE * If any of the new check­
ing moves has no legal replies, that checking move and
its ancestral reply are labelled MATE,
4. If all the replies to M have been labelled HATE, M is
labelled KATE, together with its ancestral reply; and
if all siblings of the latter have also been labelled
MATE, the label is propagated two generations further
back; and so on.
5. If M and its siblings have all been labelled NCMATE,
its ancestors for two generations will be labelled
NCMATE: and so on. (Steps 4 and 5 can b* seen to b*
simply an application of the minimax evaluation rule,
together with the alpha-beta heuristic.)
6. If any of the checking moves in the starting position
has been labelled MATE, or if a11 of them have been
labelled NCMATE, the program halts. Otherwise, it con­
tinues by repeating step 3.

Since MATER is searching for a strategy that will decide the

game Irrespective of what the opponent does, it needs no static evaluation
except MATE and NCMATE 0 Since the game is not going to continue if the
search is successful, piece values and specific criteria like center con­
trol have no significance. MATER will attempt sacrifices unhesitatingly
if they permit continuing checks of the opponent's King. If MATER does
not find a checkmating strategy, it simply halts—presumably shifting res­
ponsibility to some other (unspecified) program to suggest a move.
CIF #98 -74

The Task Environment for Checkmating

What can we say about the task environment in which MATER

operates? In the chapter on mating attacks in a standard work, Fine's

Middle Game in Chess, 129 positions are considered. In 51 of these

129 positions, checkmate can be forced by sequences of checking moves.

In five other positions, which we will discuss later, the checking
moves must be preceded by a move of a class known as one-move mating
threats. In the remaining 73 positions, checkmate can be averted

through a sacrifice of material by the defender, or the mate is not

"forced," as that term is defined by the evaluation procedure described
above*
It is only of slight interest to know that chess positions
exist in which checkmate can be enforced by a series of checking moves.
The matter becomes more interesting when some further questions are
answered. First, are these positions unusual or "pathological," or
do they actually arise in play? The answer is that they arise quite

frequently in games, although relatively rarely nowadays in games be

tween grandmasters of approximately equal strength. Second, do features
of the position exist that signal the likelihood of a checkmating com-

bination, or must, the combination be discovered by "luck"? The answer

is that reasonably reliable cues are almost always present. It is

worthwhile to look for a checkmating combination when the opponent's

Ring is exposed, or could be exposed by a capture, and when one's own
pieces are quite mobile, while the opponent's are constricted. Even a

rough measure of relative nobility would usually suffice as a signal.

GIF 98 -7S<

Thus, in the chess task environment, situations arise with

reasonable frequency in which checkmate can be enforced, and in which
this possibility is signalled by vulnerability of the opponent's King,
confined with superior mobility. But this raises the final questions

is the structure of the task environment such that the checkmate can
be discovered with a reasonable amount of search? The answer is that
it very often can. To make the answer more precise, we must define
a couple of measures of amount of search.
Consider the search tree of positions generated by a pro­
gram attempting to discover a checkmate. Figure is an example of

such a tree. We can call the number of positions on the tree that are
actually examined the sjLz<8 of the^ exploration tree. The example shows
an exploration tree of size 16. In general, however, more moves are
seen than positions are investigated—some moves put on the try list
remain unexplored, such as the replies to 1. N-K6ch in the example.
The moves seen—uninvestigated or investigated—may be called the
discovery tree. In Figure the discovery tree is of size 36 (14 checks

and 22 replies).
Another subpart of the exploration tree is the verification
tree, the branches that must be exhibited to prove that the combination
is effective. It consists of the positions resulting from the single
best move at each node for the attacker, and every legal move at each
node for the defender. The verification tree is exactly analogous to
the path leading to a theorem (i.e., a proof path) in the Logic Theorist.
It is a tree in chess, instead of a single path, because all alternatives
permitted to the defender must be tested. (In game-theory terms, the

verification tree is a strategy.) In the example of Figure , the branches

of the verification tree are crosshatched. It includes five moves and six

positions.

Achievement o£ the Mating Program

MATER, set to work with no limit on computing effort, solves com

blnations that consist of uninterrupted series of checking moves provided
that the defender has no more than four legal replies at any node in the
verification tree. In the 51 mating positions in Fine that meet these con­
ditions, MATER found solutions to 43. The program discovered check--a pro­

gram bug. It exhausted its available space in memory before finding the other
seven combinations„ Too much search was required, in these seven positions,
to find the verification tree.
Table X show certain measures of search behavior for the 43 posi­

tions in which MATER was successful: the depth of search to checkmate (D) 8
the mean siee of the verification tree necessary to prove the combination
(VT measure in moves), and the mean sise of the discovery tree generated in
searching for mate (DT measured in moves).

N (positions D VT DT VT/D DT/D VT/BT

15 2 3.5 15.5 1.8 7.8 4.4
11 3 5.4 24.6 1.8 8.2 4.6
14 4 9.9 61.5 2o5 15.4 6.2
2 5 11.0 56.0 2,2 11.2 5.1
1 8 17.0 108.0 2.1 13.5 6.4
43

MATER's Performance on 43 Positions from Fine

TABLE X
GIF #98 -77

This table provides us with a characterization of the task

environment of chess in mating*combination situations. The table shows a
fairly close correlation between depth and the sice of the verification tree
(VT/D)«»there are about two moves in the tree for each move in depth. Thus,
the tree varies more or less linearly, and not exponentially, with depth.
It is this property of mating situations that makes deep analysis possible„
The discovery tree also grows only slightly more than proportionately to
depth, and the discovery tree maintains a nearly constant ratio to the veri­
fication tree (DT/VT) of around five nodes to one.
As a result of these characteristics of the task environment,
MATER was able to find quite deep combinations with a very modest amount
of search—a sixteen-ply combination requiring a discovery tree of only 108
moves. An exponentially growing tree of depth sixteen, with only two branches
per node would have some 65,000 moves! Thus, in these forcing situations,
the simple selective heuristics of HATER achieve a radical pruning of the
tree.
A key heuristic in MATER is to always examine first those moves
that permit the opponent the fewest replies. Variants of MATER that gave
lower priority to this heuristic in determining what to explore next (in par­
ticular, depth"first exploration) were very much less successful in finding
mating combinations than the version we have described.
The fewest-replies heuristic has a dual significance: on the one
hand, it allows the opponent the fewest alternatives to find a way out of the
mating net. On the other hand, it holds down the "branchiness" of the search
tree, facilitating the discovery of the checkmate with relatively little
 L_

CIP $98 -'?8~

search. In this task environment it is not possible to disentangle these

two factors that contribute to the heuristic's power.
The fewest-replies and checking move heuristics illustrate the
concept of holding the initiative in a competitive situation. Holding the

initiative consists in taking actions that are so forceful that the opponent
must devote all his effort to replying to them--to solving the problems they
create. The side holding the initiative is able to determine the arena, so
to speak, in which the struggle is waged, and the issues on which it will be
decided.

An Extended Mating, Combinations Program

Additional insight into the structure of the task environment, and

the nature of the relevant features in it, is provided by MATER XI, and im­
proved checkmating program developed by 6. Bay lor. MATER II modifies and
extends the heuristics of MATER in three respects:
1. All checking moves are explored immediately and in depth that
can only be answered by captures or interpositions, or, in the absence of
both, allow a single King move. The routine that explores these moves keeps
a record of the moves investigated which, if it does not find an "easy" mate
by this routine, it can use later (see 2, below).
2. If any of the moves generated in step 1 are legal moves in the
initial position, they are tried even if they are not checking moves. If

the move in question, followed by a NO MOVE by the opponent, leads to imme~

diate checkmate, the move is placed in the try list. (Such a move is called
a "one-move mating threat.")
OIP $98 -79-

3. Certain other one-move mating threats are generated by

noticing moves that attack or constrict further the freedom of the op­
ponent's King.
MATER IX, tested on all five positions from Fine that necessi­
tate an initial mating threat followed by checking moves, found the five
checkmates. In two of these positions, for example, the search tree, eight
ply deep in each case, contained 37 and 38 moves, respectively. Four out
of 54 legal initial moves were considered in the one case, 3 out of 37 legal
initial moves in the second. Thus, the .search trees of MATER XI were of
the same general size as those of MATER e
The combination of the first and second heuristics of MATER II,
described above, are of particular interest to another departure (along
with the NO MOVE) from search limited to the tr«« of legal continuations.
The high~priority search of step 1, if it does not lead to checkmate, is
pursue^ for the purpose of gathering information about possible strong moves
that are not checking moves. (Chess players would call this search a "sample
variation.") The moves thus discovered may or may not be legal moves in the
initial position. Before they are added to the list of moves to be tried,
they are subjected to two teats: whether they are legal, whether they con-
stitute one-move mating threats.
Although tree search is a strong and essential component of the
mating combinations programs, it can be seen that MATER II represents a
step in the direction of a program that (1) notices features of the board,
then (2) builds strategies related to what it has noticed. The third of
the modified heuristics of MATER II is of a similar kind, even representing
CIF f 98

ft limited kind of "working backward." The reasoning might be paraphrased£

checkmates are achieved by increasing controls on the opponent's King,,
Therefore, find t;he location of the King and the squares immediately around *
him, and discover the moves that add attacks on these squares*

Functional Analysis in Chess

Our observation of the role of features and actions relevant to

those features in chess programs suggests a direction of development for
such programs that would take them very far away from tree search, but with­
out limiting them to static features. We should like to sketch out the
general lines of such an approach, which involves a more elaborate develop­
ment of the ideas of working backward, means-end relations, and keeping the
initiative.
As we have already observed, the relevance of moves to particular
features in the task environment can be described in functional terms: a
piece attacks another, or defends another, or blocks another, or pin^ another,
Certain functions (especially attacks) create problems that have to be dealt
with immediately by the opponent to prevent long-run deterioration of his
relative mobility. Moves that perform two or more such functions (double
function moves) are especially forceful, since the opponent must neutralize
both functions by his single reply. A player with the initiative can retain
it by finding a sequence of double-function moves. Most likely, the opponent
will sooner or later fail to find a satisfactory reply to one of them.
To illustrate these notions, consider forking moves by the Queen--
Queen moves that attack two undefended pieces simultaneously. The Queen is
a powerful piece because she can cove along any file, rank, or diagonal, any
 $98

number of squares. She is ?:herefore .a natural piece to consider for

attaeks°»along a fL,le and diagonal, for example, or two diagonals 5 or rank

and file. How cou;.d a program discover double attacks by the Queen? And

how could it exploit the discovery? Consider the following move°generatin|

procedure s

CREATE A QUEEN-FOiKING GOAL

(Discover two opponent c s pieces and a square
•:rom which both can be attacked.)
FIND UNSATISFIED CONDITIONS FOR FORK
(/ire the pieces undefended; ig Queen on
forking square, etc.?)
GENERATE MOVES THAT HtS<L SATISFY UNSATISFIED CONDITIONS
(Capture defender of target pdeca; move
Queen to square, etc.)
KVA&OATE MOVES
(Return to the general move-evaluation procedure.)

The firs£ step in this program,, creating the goal, can be incor­
porated SB part of the noticing program that sises up a position. The final

step cam be carried out by an analysis and evaluation program like that al«
ready contained in the NSS program. The second and third steps involve seals-

thing newc They are capable of generalization in a way that shows they hsve

application beyond the chess task environmentJ

GENERATE A GOAL SPECIFICATION, ABSTRACTING IT ERC&i

THE PRESENT SITUATION
FIND DIFFERENCES BETWEEN GOAL AND PRESENT SITUATION
GENERATE ACTIONS RELEVANT TO REMOVING DIFFERENCES
EVALUATE THE ACTIONS, AND APPLY IF SATISFACTORY

But this is a quite general scheme for means~end analysis ° A

program in search for a Queen fork in this way is not simply applying its
special knowledge of chess; it is applying to chess a general capability
I£ #98

for reasoning in t firms of means and endg= In the next chapter we shal.t

show how a £ask»independent general problem«°8olving program can be con­

structed on tha b&uis of means-end analysis.
To give f:he Queen- forking „ or other means - end program an ability

to retain the initiative 9 we simply incorporate it in a recursive scheme.

in which the relevant actions, to be evaluated as satisfactory, tcust be
double^function actions. For example y suppose a move is discovered that

will take the Queen to a square from which she can then move to the forking
square. The former move w&jld only be acceptad if it also performed antsthe
function—attacked another undefended piece, say 0
The important characteristics of programs of this kind &re (1)
that they operate not in terms of f,he concrete representation of ths task
environment, but in terms of a functional description of some aspects of
that environment; (2) that they work backward, by generating goals,, some
but not all of whose specifications are satisfied by the present situation ?
then trying to realize those goals; (3) that they employ a very general
scheme for means-end analysis this is not special to the tfisk environment,

CONCLUSION

This chapter does not call for a lengthy summary, since it con­

tains eummaries ot the main sections. We wish merely to recall the central

themes <>
The theme of the first sections of the chapter was the tradeoff
between search and selectivity in finding solutions to problems in complex
task environments <• We examined a sequence of chess-playing programs that
 &

represent a general historical trend from the examination of very large

numbers of positions in the search tree to much more selective search
based on specific chess heuristics about: what to notice and what to do
about what is noticed. The evidence gives at least soina reason to believe

that the future lies in the direction of even greater selectivity, and that
the power of selectivity owes nothing to any very special characteristics
of the chess environment-"it is a prime consideration in handling success
fully any very large search tree.
In the latter portions of the chapter., we introduced the notions
of means and ends, of functional ar-ialysis, and several others related to
them. On the basis of these, we began an exploration of ways of viewing
problem-solving tesks like chesa playing in terms other than tree search.
We saw that there are already important elements 5,n existing cheES pro­
grams that go beycnd the tree^seerch paradigm, and that many new possibil­
ities for problem-solving programs lie in these slightly-explored directions.
Our task in the next chapter will be to follow up our introductory

remarks about means-end analysis by describing a problem-solving program

that makes important use of this concept. In so doing we will begin oar

transition fram the discussion of the task environment to the analysis of

human problem solving.