NATIONAL PHYSICAL LABORATORY

S Y M P O S I U M No.
10

Mechanisation of Thought Processes

Proceedings of a Symposium held at the National Physical Laboratory on 24th, 25th, 26th and 27th November 1958

VOLUME I

L O N D O N : HER M A J E S T Y ' S S T A T I O N E R Y O F F I C E
1959
(Reprinted 1962)

SESSION 3
PAPER 6

PANDEMONIUM: A PARADIGM FOR L E A R N I N G

by

D?. 0. G. S E L F R I D G E

Cllver G. Selt'rlci~e was born I n London 10 M?;; 15'C He studied 2 t the Massachusetts Institute of Tfcci-iriology frorr. 192-1945, returning post~ - r a r i . d a t c j . y ??or I94?-1950. Alter 2 y e a r s a t S g n a l Corzs Laticrstorl e s at Fort Mor.mouth. ! Ncv Jersey, he 2 o l n e d Lincoln Labora- t o r i e s In GX'JD 34, Corr'iunicatlon Techniques, of W:-.lch he Is now Group Leader.

PANDEMONIUM:

A PARADIGM FOR L 7 A R M M

DR. 0. G SELFRIDGE .

INTRODUCTION WE a r e proposing h e r e a model o f a process which we claim can a d a p t i v e l y

improve i t s e l f t o handle c e r t a i n p a t t e r n r e c o g n i t i o n problems which cannot be adequately s p e c i f i e d In advance. Such p r o b i e r s a r e u s u a l when t r y i n g t o b u i l d a macnine t o I m i t a t e any one of a very l a r g e c l a s s of human d a t a processing techniques. A speech t y p e w r i t e r I s a good example of sonething chat very many people have been t r y i n g u n s u c c e s s f u l l y t o b u i l d f o r some time. W do n o t suggest t h a t we have proposed a model vhlcn can l e a r n t o e typewrite from merely h e a r i n g speech. Pandemonium does n o t , however, seem on paper t o have the same k i n d s of l n n e r e n t r e s t r i c t i o n s o r i n f l e x i b i l i t y t h a t many previous proposals have had. Trie b a s i c motif Dehind o u r model i s the n o t i o n of p a r a l l e l processing. This i s suggested on two grounds: f i r s t , i t i s o f t e n e a s i e r t o handle d a t a I n a p a r a l l e l nanner, and, Indeed, I t i s u s u a l l y t h e more " n a t u r a l " manner t o handle i t in; and, secondly, i t I s e a s i e r t o modify an assembly of quasi-independent modules than a machine a l l of whose p a r t s i n t e r a c t immediately and i n a complex way. We a r e n o t going to apologize f o r a frequent use 01 antnropomorphic o r ~ i o m o r p h i cterminology., They seem t o be u s e f u l words t o d e s c r i b e our notions. What we a r e describlrig i s a process, or, r a t h e r , a model of a process. W s h a l l no? d e s c r i b e all t P n reasons thac l e d t o i t s particular formulation, e out we s h a l l give some r e a s o n s f o r noping t h a t i t does lr f a c t possess t h e f l e x i b i l i t y and a d a p t a b i l i t y t n a t we a s c r i b e t o i t .

THE PROBLEM ENVIRONMENT FOR LEARNING

PandeTionium i s a -nodel hrhicn vie nope can l e a r n t o recognize p a t t e r n s which nave n o t been s p e c i f i e d . W nean t n a t i n cue f o l i o Jing sense: we p r e s e n t t o e the model examples of p a t t e r n s taken from some s e t of theri, each time I n f o m i n g tne nodel which p a t t e r n we n i d j u s t presented. Then, a f t e r some time tne model guesses C o r r e c t l y which p a t t e r n has J u s t been presented beft - d

we inform i t . For a l s r g e c l a s s of p a t t e r n r e c o m i t l o n ensembles there has never e x i s t e d a n y adequate w r i t t e n o r s t a t a b l e d e s c r i p t i o n of t h e d i s t l n c t i o n s between the p a t t e r n s . The o n l y requirement '/le can p l a c e on our modal I s t h a t ;-;e want i t t o benave i n tC.5 same '::ay t h a t Xen observably behave In. In an a b s o l u t e sense t h i s I s a v e r y u n s a t i s f a c t o r y d e l ' i n l t i o n of any task:, but i t may be a p p a r e n t t h a t i t I s the ':;a; In which most t a s k s a r e defined f o r most men. Lucky is he whose job can 'oe e x a c t l y s p e c i f i e d i n words withouc any ambiguity or n e c e s s a r y Inferences. The example we s h a l l I l l u s t r a t e i n some d e t a i l is t r a n s l a t i n g from manually keyed Morse code Into, say, t y p e w r i t t e n messages. Now i t I s t r u e t h a t when one l e a r n s Morse code one l e a r n s t h a t a dash should b s e x a c t l y t h r e e times the l e n g t h of a dot and s o on, but i t t u r n s o u t t h a t cl-.ls I s r e a l l y m o s t l y I r r e l e v a n t . What m a t t e r s i s only what tne v a s t army of people who u s e Morse code and with whom one I s going to have t o communicate understand and p r a c t i s e when they u s e i t . I t t u r n s o u t t h a t t h i s I s n e a r l y alw?ys v e r y d i f f e r e n t from school book Morse. I n the same way the only adequate d e f i n i t i o n of the p a t t e r n of a spoken word, o r one hand-written, must be In terms of the consensus of the people who a r e u s i n g i t . W use the term p a t t e r n r e c o g n i t i o n In a broad sense t o include n o t only e t h a t d a t a processing by which images a r e assigned t o one o r another p a t t e r n . in some s e t of p a t t e r n s , b u t a l s o the p r o c e s s e s by which t h e p a t t e r n s and d a t a proc,essing a r e developed b y the organism .er machine; we g e n e r a l l y c a l l this l a t t e r "learningffe

PANDEMONIUM,

IDEALIZED AND PRACTICAL

W first c o n s t r u c t an i d e a l i z e d pandemonium (figa 1 ) . Each p o s s i b l e e P a t t e r n o f the s e t , r e p r e s e n t e d by a demon In a box, computes h i s s i m i l a r i t y w i t h t n e image simultaneously on view t o a l l o f them. and g i v e s an output depending monotonlcally on t h a t s i m i l a r i t y . The d e c i s i o n demon on t o p makes a choice of t h a t p a t t e r n belonging t o the demon whose output was the l a r g e s t .

* This

I s an e x a c t c o r r e l a t e of a c c m u n i c a t . l o n s system ,;.herein given a r e c e i v e d '() message M W and a n- int tier of p o s s i b l e t r a n s m i t t e d messages !. ! , t h a t . " 1s chosen, t h a t I s , deemed t o have been t r a n s m i t t e d , ~ i h l c hminimizes 2( ~ v c T ) - ~ ~ I~ d II. ) (Such a procedure I s optimum under c e r t a i n c o n d i t i o n s ) . T h i s I n t e g r a l Is, a s I t were, t h e s q u a r e of a d i s t a n c e in a s i g n a l phase space f i e . 2 - and t h u s t h a t trans-riltte'i n e s s a g e I s s e l e c t e d t h a t Is most s i n l l a r t o t h e r e c e i v e d one.

PICKS THE LARGEST OUTPUT FROM THE

COGNITIVE DEMONS, WHO INSPECT THE

CATA OR IMAGE.

Each demon may, f o r example, be assigned one l e t t e r of the alphabet, s o t h a t the task: of the A-demon I s t o shout a s loud of the amount of 'A-nessl t h a t he s e e s In the image.* Now i t w i l l u s u a l l y happen t h a t w i t h a reasonable c o l l e c t i o n ol' c a t e g o r i e s l i k e the l e t t e r s of the a l p h a b e t - the conputal i o n s cerformed by each o f t h e s e i d e a l c o g n i t i v e demons w i l l be l a r g e l y t h e same. In many Instances a p a t t e r n I s n e a r l y e q u i v a l e n t t o some l o g i c a l function of a s e t of f e a t u r e s , each of which is i n d i v i d u a l l y comon t o perhaps s e v e r a l p a t t e r n s and whose absence is a l s o comon t o s e v e r a l o t h e r patterns.? t h e r e f o r anend our i d e a l i z e d Pandemcnium. The amended v e r s i o n fie. 3 - has scme profound advantages, c h i e f among which is i t s suscepti b i l i t y t o t h a t kind of a d a p t i v e self-improvement t h a t I c a l l learning. The d i f f e r e n c e between f i g . 1 and f i g . 3 is t h a t the common p a r t s of the computations t h a t each c o g n i t i v e demon c a r r i e s o u t i n f i g . 1 have in f i g . 3 been a s s i g n e d i n s t e a d t o a h o s t of subdemons. A t t h i s s t a g e the organizat i c n has four l e v e l s . A t the bottom the d a t a demons s e r v e merely t o s t o r e and p a s s on the d a t a . A t the n e x t l e v e l the computational demons o r subdemons perform c e r t a i n more o r l e s s complicated computations on t h e d a t a and pass t h e r e s u l t s of t h e s e up t o the n e x t l e v e l , the c o g n i t i v e demons who weigh the evidence, a s i t were. Each c o g n i t i v e demon computes a s h r i e k , and from a l l the s h r i e k s t h e h i g h e s t l e v e l demon of a l l , the d e c i s i o n demon, merely s e l e c t s the loudest.

THE CONCEPTION OF PANDEMONIUM

W cannot a b I n i t i o know t h e i d e a l c o n s t r u c t i o n of our Pandemonium. W e e t r y t o a s s u r e t h a t I t c o n t a i n s the seeds of self-improvement. Of the f o u r l e v e l s i n fip. 3 . a l l b u t the t h i r d , t h e subdemons, which compute, a r e s p e c i f i e d by the task. For the t h i r d l e v e l , t h e r e f o r e , we c o l l e c t a l a r g e number of p o s s i b l e u s e f u l f u n c t i o n s , e l l m l n a t l n g a p r i o r i Only those which could n o t conceivably be r e l e v a n t , and make a reasonable S e l e c t i o n of the o t h e r s , b e i n g bound by economy and space. We then guess reasonable weights f o r them. The behaviour a t t h i s point may even De a c c e p t a b l y good, b u t Usually i t must be improved by means of the a d a p t i v e changes we are about t o discuss.

I t I s s o s s l b l e a l s o t o p h r a s e I t s o t h a t t h e A-denon I s concutlng the d i s t a n c e I n sone phase of t h e l r a g e from scne I d e a l A; I t seems t o m 'Jnnecessarlly p l a t o n i c t o p o s t u l a t e the e x i s t e n c e o r ' I d e a l 1 r e p r e s e n t a t i v e s o f l a t t e r n s , ana, Indeed, t h e r e a r e o f t e n good r e a s o n s f o r n o t d o l n s so. / See, f o r example, J e r o n e Bruner, % s t u d y of Thlnkl~Ig*,

COGNITIVE

DEMONS

COMPUTATIONAL DEMONS

DATA OR IMAGE DEMONS

The E v o l u t i o n of Pandemonium There a r e s e v e r a l k i n d s of adaptive changes which we w i l l d i s c u s s f o r our 7 Pandemonlim. They a r e a l l e s s e n t i a l l y very s i m i l a r , b d t tney nay be p r o g r m e a and discussed separately. The first n a y be c a l l e d "Feature Weighting". Altnough we have n o t y e t s p e c i f i e d what t n e c o g n i t i v e denons conpute, the s o l e t a s k a t present is t o add a weignted SLCT o trie o u t p u t s o f a l l t3e computational demons o r s-Jbdenons; the weightings w i l l of course d i f f e r for

Flg.4.

F i r s t hill c l i m b i n g technique: p i c k v e c t o r s a t r a n d = ( p o i n t s In t h e s p a c e ) , s c o r e tre'n, a r d s e l e c t t h e one t h a t s c a r e s h i g h e s t .

the d i f f e r e n t c o g n i t i v e demons, b u t the weightings w i l l be the only d i f f e r e n c e between them. Feature weighting C o n s i s t s of a l t e r i n g t h e weigy.ts assigned to t h e subdemons by t h e c o g n i t i v e demons s o a s t o maximize the s c o r e of the e n t i r e 7 Pandemonium. How then can we do t h i s ?

The Score What we mean b y the s c o r e nere I s how well the machine i s doing tne tasA we want i t t o do. This presumes t h a t we a r e monitoring the machine and t e l l i n g i t when I t makes an e r r o r and s o on, and f o r the r e s t of the d i s c u s s i o n we s n a i l be assuming t'qat we have a v a i l a b l e some such running s c o r e . Now a t sone p o i n t we s n a i l be very i n t e r e s t e d i n Having the machine r J n .-.'Ithout t z a t kind of d i r e c t s u p e r v i s i o n , aKd the question n a t u r a l l y a r i s e s w r e t r e r the rnacr-ine car. m e a n i n g f u l l y monitor i t s own perforinance. Je answer t h a t question t e n t a t i v e l y yes, b ' ~ td e l a y d i s c u s s i n g i t t i l l a l d c e r section.

~ l g - 5 . secozd n l l l - c l l m b l n g techniiiue: p i c k vectors until o f the--, (lpuxltter 4) outscores the previous ones. Then take s h o r e i ardoj-l steps, retracing any that decrease the s c o r e .

Feature ifeightang and Hi. Ll-Cl

t'mbing

The o u t p u t o f any c o g n i t i v e demon i s

so t h a t the complete s e t o f w e i g h t s f c r a l l t h e c o g n i t i v e de-nons over a i l the subdemons I s a v e c t o r ;

Now f o r some (unknown) s e t of weights A, t h e behaviour of t h i s whole Pandenonimi I s optimum, and t h e problem of f e a t u r e weighting I s t o f'ina

MAIN PEAK

KS

Flg.6.

General s p a c e showing f a l s e p e a k s . One of t h e f a l s e peaits ~ 3 1 1 e I s o l a t e d f r o " ^ t h e n a i n o? t r u e peakt

t h a t s e t . This l a y be d e s c r i b e d a s a h i l l - c l i m b i n g ~ ? r r t l e m . W have a space e e (of A) and a function on the space ( t h e s c o r e ) , which w piay consider an a l t i t u d e , and whicn \:e ,vlsh t o maximize by a proper s e a r c h through A. one p o s s i b l e technique I s t o s e l e c t weighting v e c t o r s a t random, s c o r e tnem, and f i n a l l y t o s e l e c t the v e c t o r t n a t s c o r e d h i g h e s t ( s e e f\g. 4). I t w i l l u s d a l l y , however, t u r n o u t t o be p r o f i t a b l e to t a k e ~ d v a n t a g e of the c o n t i n u i t y p r o p e r t i e s of tne space, which u s u a l l y e x i s t In some sense, i n t r e following way: s e l e c t v e c t o r s a t random u n t i l you f i n d one t n a t s c o r e s perceptiblymorfc tran the otrers, ?-an t h i s point take s n a i l randor s t e p s I n a l l d i r e c t i o n s ( t h a t I s , add small random v e c t o r s ) m t i l you f i n d a d i r @ c t ' o n t h a t improves your score. When you f i n e sucn a s t e p , take i t and r e p e a t the process. Tnis I s I l l u s t r a t e d In fq., and i s the case of a b l i n d man 5 t r y i n g t o climb a h i l l . There may ce, of course, TanJ f a l s e peaks on wnicn one may f i n d oneself trapped In sdcn a procedure {fig. E).

' E e prol-'le-";of r a i s e p e x s in searc.'-.in%tec.'.nIc.ues I s an o l d an;; f a n l l i a r one. In g e n e r a l , one may hope ti-.a: in spaces of v e r y ::i3;h dlxens i o n a l i t y t ; ~ eInterdependence c : m e componer.ts and the score i s so g r e a t a s t o inE.<e v e r y QnllKely L!".? e x i s t e n c e of f a l s e peaks c o n c l e t e l y I s o l a t e d from Cue m i n o r tree peak;. I t n u s t be r e a l i z e d , however, t h a t :rils is a p u r e l y e x p s r l - ~ . e r . t a ~ ' - i e s t l o nt h a t h a s to be answered s e p a r a t e l y :or every l hill-cl1rn~~i1-g i t u a t i o n . I t 50:s tl-irn out In l-.ill-clin3ing siti-i?tlor.s tl-.at s the choice of s t a r t i n g point I s octen very important. The main peak may b e very prominent, but u n l e s s I t n a s wide-spread foot-3111s i t nay r a c e a very long t i n e before we e v e r 3ezin to gain a l t i t u d e . T'nis w.vf b e described a s or.? of the proolems of tr?.lnirig, n m e l y , t o encourage the nac'nine o r organism t o s e t enough on the f o o t - 5 i l l s so t h a t small changes i n h i s Darar.etprs d i l l produce n o t i c e a b l e Improvenent i n h i s a l t i t u d e o r score, One can d e s c r i b e l e a r n i n g s i t u a t i o n s where n o s t of the r d i f f i c , ~ l t yo: toe t a s k l i e s i . f i n d i n g a s y ',-:ay o f irriproving onef s score, s x n a s l e a r n i n g t o r i d e a u n l c y c l e , where I t takes longer t o s t a y on f o r a second than I t does t o i m ~ r c v et n a t one second to a minute; and o t h e r s where I t is easy t o do a l i t t l e well and v e r y &rd t o do very well, sucn a s l e a r n i n g to p l a y c5ess. I t i s a l s o t r u e t h a t o f t e n tne n a i n geai? i s I ~ l a t e a ur a t h e r than an i s o l a t e d s?i;<e That i s t o say, o p t i n a l behaviour 01' the inecLanIsm, once re?-ched, pi5y be r a t h e r i n s e n s i t i v e t o t h e change of some of the parameters,
Subdemon S e l e c t i o n

The second kind of a d a p t i v e change t h a t we wish t o i n c o r p o r a t e i n t o o u r Pandemcnluii I s subdemon s e l e c t i o n . A t the conception c f our demoniac assembly vre c o l l e c t e d sonewhat a r b i t r a r i l y a l a r g e nunber of subdemons :;'rile?. we guessed w w l d be u s e f u l and a s s i g n e d thex weights a l s o a r b i t r a r i l y . The f i r s t adaptlve c?ange, f e a t u r e weighting. optimlze-i t h e s e weights. but we have no assurance a t a l l t h a t the p a r t i c u l a r subdemons we s e l e c t e d a r e good ones. Sutdemon s e l e c t i o n g e n e r a t e s new subdemor.~ f o r t r i a l and e l i m i n a t e s i n e f f i c i e n t ones, m a t , i s , ones t h a t ao n o t much h e l p improve the score. W gropose t o do t h i s i n i t i a l l y by two di:f'.ferent techniques, which may e be c a l l e d "mutated f i s s i o a t l and Nconjugation". The f i r s t p o i n t t o note i s t h a t I t I s p o s s i b l e t o a s s i g n a worth to each of the s u b d e m s . I t may t e done in s e v e r a l bays, and we nay, f o r example, w r i t e the worth I$ of t?.e i t h demon

s o t h a t tine worthy demons are those hose o u t p u t s a r e l i k e l y t o a f f e c t most s t r o n g l y the d e c i s i o n s ;nade.

W assume t h a t f e a t u r e weighting h a s a l r e a d y run s o long t h a t the e behaviour of the machine h a s been approximately optimized, and t h a t s c o r e s and worths of xiachine and i t s demons have been obtained. First we e l i m i n a t e those subdemons w i t h low worths. Next we g e n e r a t e new subdemons by mutating the s u r v i v o r s and reweighting the assembly. A t p r e s e n t we p l a n t o p i c k one subdemon and a l t e r some of h i s parameters more o r l e s s a t random. This will u s u a l l y r e q u i r e t h a t we reduce the subdemon himself t o some c a n o n i c a l form so t h a t the random changes we i n s e r t do n o t have the e f f e c t of r e w r i t i n g the program s o t h a t i t w i l l n o t run o r s o t h a t i t w i l l e n t e r a c l o s e d loop without any hope o f g e t t i n g o u t of it.* Besides muta Led f i s s i o n , we a r e proposing another method of subdeinon improvement c a l l e d ltconjugationll. o u r purpose here i s two-Sold: f i r s t t o provide a l o g i c a l v a r i e t y in the f u n c t i o n s computed by the subdexions, and. secondly, t o provide l e n g t h and complexity In them. What we do i s t h i s : glven two ' u s e f u l ' subdemons. we generate a new subdemon which is the continuous analogue of one of t h e t e n n o n t r i v i a l b i n a r y two-variable f u n c t i o n s on them. For example, the product of two subdemon output S, corresponding t o t h e l o g i c a l product, would Suggest trie simultaneous presence of two f e a t u r e s . The t e n n o n - t r i v i a l such f u n c t i o n s a r e l i s t e d In Table I.

A . BVwA . 3

A.?13v-A

Table 1. Non-trivial b i n a r y functions on

two v a r i a b l e s .

CORRECT DECISION
\ '

(FROM ME)

COMPUTING

SUBDEMONS

LETHE

EVEN -NUMBERED DURATIONS ARE M R S AK ODD- NUMERED DURATIONS ARE SPACES

2S

AS we mentioned before, the e n t i r e n o t i o n of Pandemonium ,-iids conceived C L i r a c t i c a l way o f a u t o m a t i c a l l y improving data-processing f o r p a t t e r n

recognition. our i n i t i a l model t a s k I s to d i s t i n g u i s h :dots and dashes I n manually keyed Horse code, so t h a t o u r Pandemonium can be I l l u s t r a t e d In fig. 7. Vote that. the f l s . c t i o n s and behaviour of a l l de.nons have 'ceen s p e c i f i e d except f o r t h e computing subdemons. W s h a l l r e i t e r a t e tnose e spec1 f i cations. (1) The d e c i s i o n demon1s output I s ' d o t 1 o r ' d a s h ' according a s tr.e d o t demon's output i s g r e a t e r o r l e s s than ine dasfi demon's. ( 2 ) The cognitive demons, d o t and dash, each compute a ',deignted s m of u the o u t p u t s of some 150 computing subdeinons. I n i t i a l weights we 'nave assigned a r b i t r a r i l y , b u t , we hope, reasonably. (5) The data-handling demons r e c e i v e d a t a In me form of d u r a t i o n s , a1 t e r n a t i v e l ; c ? marks and spaces, and they pass them ?own the l i n e .

OUT PUT

1-5
' 0

/^

Tne computing sxbdernons a r e c o n s t r u c t e d Tram o n l y a v e r y few o p e r a t i o n a l functions, which a r e c a r e f u l l y non-binary. For example the subdemons d = d p and d o d r , hive t h e i r o u t p u t s shown i n fzg.3. The operational, functions follow 1 ) l = ' . This function computes the degree of e q u a l i t y of some s e t o l v a r i a b l e s ( s e e ftg, 5). ( 2 ) ' ', l , compute the aegree to which some v a r i a b l e i s l e s s than o r 1s g r e a t e r than some o t h e r v a r i a b l e (see f i z h8). ( 3 ) 'm&, ' m a x ' , compute the degree t o which some v a r i a b l e I s the l a r g e s t of an a r b i t r a r y s e t of v a r i a b l e s o r an a r b i t r a r y s e t of consecutive variables, s t o x 1222 degree t o ~.&iIcF. :i;e izh d ~ i ~ a ~ hl o r been s ~ (4) '(lL , t A t I d e n t i f i e 6 as a dot o r dash. ( 5 ) "W computes 2n average o? socie s e t 01 v a r i a b l e : , (S) ' , ' / I i s a family of t r a c k i E g means. For example, i t ~'iis::.t co;r@lite
L

CONCUJSI ON

',&at I s h a l l present a t the meeting In November w i l l be the d e t a i l s of the p r o g r e s s of Pandemonium on the Yorse code t r a n s l a t i o n problem. The i n i t i a l emblem we have given the machine I s t o dIstln:?uIsh d o t s and dashes. When the behaviour of the machine has Improved I t s e l f to the p o i n t where l i t t l e f u r t h e r Improvement seems t o be occurring, we s n a i l add t h r e e more c o g n i t i v e demons, the symbol space, the l e t t e r space, and t h e w r d space. Presumably a f t e r sone f u r t h e r time t h i s new Pandemonium will s e t t l e down t o some unlriprovable s t a t e . Then we s h a l l r e p l a c e the s e n i o r o r decision-making d e m n '.-,'l a row of some f o r t y o r s o c h a r a c t e r demons with a new declsionth inaKing demon above them, l e t t i n g the new c o g n i t i v e demons f o r the c h a r a c t e r demons use a l l the I n f e r i o r demons, c o g n i t i v e and otherwise, f o r t h e i r Inputs. I t I s probably a l s o d e s i r a b l e t h a t previous d e c i s i o n s be a v a i l a b l e f o r present d e c i s i o n s , so t h a t a couple of new f u n c t i o n a l o p e r a t i o n s might be added. There need be l i t t l e concern about 10giCal C i r c u l a r i t y , because we have no requirement f o r l o g i c a l c o n s i s t e n c y , b u t a r e merely seeking agreeable Morse t r a n s l a t i o n . How much of m e whole Drogram w i l l have been run and t e s t e d by Nover/oer I cannot be s u r e of. A t the p r e s e n t ( J u l y ) we have had some f a i r t e s t i n g o f h i l l - c l i m b i n g procedures.

ACKNOWLEDGEMENT
I should l i k e t o acknowledge t h e v a l u a b l e c o n t r i b u t i o n s o f d i s c u s s i o n s with many of my f r i e n d s , Including e s p e c i a l l y M. Mlnsky, U Neisser, . F. Frick and J. L e t t v l n .

DISCUSSION ON THE P A P E R BY DR. 0. G. SELFRIDCrF

DR. J. W. BRAY: May I a s k Dr. S e l f r i d g e what n e t h i n k s of t h i s approach t o h i s problem? Let X = d u r a t i o n o f t n e l a s t s i g n a l , which may oe a d o t o r a dash, a s i g n a l space, l e t t e r space o r ward Space. Let y = 2, i f i n f a c t i t was a dasn, 1 I f dot, 0 i f s i g n a l space, -1 I f l e t t e r space and -2 i f word space. .,3~ To form the polynomial:

t a k e a number of o b s e r v a t i o n s , a s h e suggests, and l e t tne machine l e a r n t h e code by determining t h e c o e f f i c i e n t s Aoetc. by simple c u r v i l i n e a r regression. The d u r a t i o n o f p r e v i o u s and subsequent s i g a l s ana the i n t e r p r e t a t i o n given t o p r e v i o u s s i g n a l s could be added a s f u r t h e r v a r i a b l e s on the r i g n t hand side.

HR. C. STRACHEY: A t Lne end o f me Paper you promise u s some f u r t h e r information about t h e l a t e s t s t a t e of t h e programme. Could you l e t u s know what t h i s is?
DP- J. McCARTHY; I would l i k e t o speaK b r i e f l y about some of tne advantages of tne pandemonium model a s an a c t u a l mooel o f conscious oehavlour. In observing a brain, one should maKe a d l ~ t i n ~ t between t h a t a s p e c t of the i~n behaviour which I s a v a i l a b l e consciously, and tflose Denavlours, no doubt e q u a l l y important, but whicn proceed unconsciously. I f one conceives of the Drain a s a pandemonl'.~n- a r o l l e c t l o n o f demons - pernaps vhat I s going on within the demons can be regarded a s t h e unconscious p a r t of thought, and what the demons a r e p u b l i c l y s h o u t i n g f o r each o x e r to hear, a s the conscious p a r t o f thought.
S 'pandemonium' h a s a c e r t a i n fa~mlly resemblance to a c l a s s o f mechanism considered i n some e a r l i e r papers (ref. l ) (tnough I had never suspected i t s demonic i V n $ l l c a t i o n s . ' ) . In one of these, ( r e f . 2) a f t e r d i s c u s s i n g the g e n e r a l p r i n c i p l e t n a t you p e n a l i s e t h e unsuccessful, I p o i n t e d c u t t n a t the amount of i r & f o r n a t i o n

DR. D. M. MACKAY: Dr. Sel f r i d g e '

1 MACK.iY, D. M. M e n t a l i t y In Machii-ies. f'rcc. . A r i s t . Soc. S u p f t . , 1932, 61. 2 KACKAY, D. M. The E p i s t e m o l ? g l c a l Proclem f o r ~ u t o m a t a , ea. J. McCarthy and . C . E. Shannan Automata S t d e s , ??-inceton (1955). S e e a l s o &it. J . ?syc^iol., 1956, 47, 30 and Advancement of Science, 1956, 392.

p e r t ~ i c (~ ol use FT. S e l f r i d = e is neta^iicr) i s , e r / m a l l u n l e s s the t p r o s a s i l i Z Y of s u c c e s s arc f a i l u r e a r e w a l . I f you ^ a v - a sb s t e ? ,mere t n e r e a r e a v a s t n - i i l e r c f ~ c s s i ~ i l l t i eco be e l i n i n d t e a t-y r a t n e r fee-dess s t r i a l and e r r o r , tnt-n of coarse f a i l u r e occurs "idcn nore o f t e n than success. Tne s o l u t i o n I s J L e s t e d was t o for-n a Kind of s y n d i c a t e d l e a r n i n g p r o c e s s I n wnlcn a t f i r s t l a r - t nufl'sers of elements, d e s t i n e d e v e n t u a l l y f o r independence, snouln n s c n n ~ l e dt o g e t h e r s o a s to reduce t n e d i v e r s i t y of t response. A hand, f o r example, n l ~ l n o t a t f i r s t nave M C ^ f i n :er s e p a r a t e l y c o n t r o l l a 3 l e 1 b u t c & ~ l cworc clumsily a s a bdnole. I n t h a t d a ~ ,you can g.e.r e a t l y decrease t r e amount of groping & i c h i s n e c e s s a r y b e f o r e a s u c c e s s f u l

--

act
e

Gay De l e s s . On t h i s p r i n c i p l e , i f pursded t o trie l i m i t , even the e a r l i e s t Trials c o ~ l d nave a non-ned.lgic% chance o f s u c c e s s , (tt-ough s u c c e s s i n a e r 1 small way^). Given t h i s easy s t W t , s i n p l e s e l f - o r g a n i s i n g s u b r o u t i n e s can b u i l d up f a i r l y quickly. A s they i n c r e a s e t h e i r number ana t h e i r success, however, t h e I d e a I s t h a t the couplings between elements should g r a d u a l l y t e d i s s o l v e d t o i n c r e a s e toe complexity of t h e problem. I f ~ O Jkeep uhe complexity i n c r e a s i n g s t e p b,y s t e p with t h e degree o f development o f s u c c e s s f u l I n t e r n a l n a t c n i n g sub-routines, tnen f u l l y a d a p t i v e behaviour can oe enornously 'nore quickly developed than i f m e s y s t e n s t a r t s vvith t n e y f u l l r e p e r t o i r e to oe explored. M question I s why Dr. S e l f r i d g e n a s n o t i n c o r p o r a t e d t h i s p r i n c i p l e i n n i s l~andemonium', s o t n a t eacn ' k i c ~ ' ~oula nave sometnin," n e a r e r CO one oli, of i ~ i f o n n a ~ i oin s t e a a of an - i l ~ i i o s ~ n n e ~ lgli b l e f r a c t i o n .

C ORICE \ w r i t t e n c o n t r l o u t l o n \ " Tnls i s a very I n t e r e s t i n g anc s t i n i ~ l a t i n gpaper. I nave one coniflent t o m x e , and t h a t i s on t h e discuss i o n of "Feature Wei&tIne and dill-ClirnDingH. I t h i n k t h a t in t h i s d i s c u s s i o n the a u t h o r h a s not Drought cut, one important d i s t i n c t i o n between types of flhill-climbingfl problems - t h a t between determinate and s t o c h a s t i c problems. Whereas i n a determinate problem Lne " h i l l u i s d e f i n e d by a s i n g l e f u n c t i o n o f many v a r i a b l e s
38, ?

i n a s t o c h a s t i c problem I t I s d e f i n e d by the mean v a l u e of a number of functions:

and a i s a random v a r i a b l e whose value depends on t h e p a r t i c u l a r t r i a l made. Now I would have thought t h a t n o s t o f the more i n t e r e s t i n g p a t t e r n recogni t i o n problems a r e e s s e n t i a l l y s t o c h a s t i c : t h e obj e c t i v e i s t o maxe

a macnine t n a t w i l l o b t a i n a s n i g n a p r o p o r t i o n as p o s s l . - ' l e o r c o r r e c t answers t o a s e r i e s o f q u e s t i o n s Itwhat i s t h i s p a t t e r n ? " r e f e r r i n g t o a r3ndoia s e r i e s o f p a t t . e r n s . Tile a u t n o r h a s felven an e x c e l l e n t account o f some of trie d i f f i c u l t i e s o f d e t e r m i n a t e h i l l - c l i n b i n s and o f t n e t e c h n i a u e s f o r overcornin& thel", b u t s t o c h a s t i c h i l l - c l i m b i n ^ : p r e s e n t s a d d i t i o n a l problems, and I thinK t h i s :,';ay become v e r y a p p a r e n t when a pandemonium is ~ u i l t o a e a l w i t n a p r a c t i c a l task. I n p a r t i c u l a r , two problems which w i l l n e e d Invest!.-ation are: (1) how l a r g e a s a n p l e o f t r i a l v a l u e s f ( x ) and belonging to two p o i n t s x and y i n -.the v e c t o r s p a c e b e i n g e x p l o r e d , w i l l be needed t o o b t a i n a s a t i s f a c t o r y e s t i m a t e o f [ f ( x ) - fi)]f o r t h e p u r p o s e s o f hill-cllrfAIng? (11) hovi iruch l o n g e r w i l l a p a r t i c u l a r s r c h a s ti c h i l l - c l i r l ' s i n g p r o c e d u r e t a k e than t h e c o r r e s p o n d i n g d e t e r m i n a t e p r o c e d u r e , a n a w i l l t h i s r a t i o prove t o be t o o l a r g e , i n some p r a c t i c a l i n s t a n c e s , t o a l l o w t h e evolut i o n a r y development o f t k e pandemonium t o taKe p l a c e ? I s h o u l d be v e r y I n t e r e s t e d t o l e a r n i f t h e a u t h o r h a s c o n s i d e r e d t h e s e problems and developed any s o l u t i o n s . Tne problem o f s t o c h a s t i c h i l l cllrnbine; Is o f P r a c t i c a l i n t e r e s t o u t s i d e t h e c o n t e x t o r QiIs p a p e r , f o r I n s t a n c e i n c o n n e c t i o n w i t h t n e e v o l u t i o n a r y o p e r a t i o n of c n e n i c a l p r o c e s s e s .

fdy),

G SELFRIDGE ( I n r e p i / ^ ) : S i n c e I have been * o r k i n on Morse code, . rihich I an doing i n a d d i t i o n t o wording on l e a r n i n g , I nave i-net probaoly 50 p e o p l e and when I say t h a t I am working on a macnine t o do m a n u a l l y keyed Morse code, they s a y "1 w i l l t e l l yoa how t o do m a t ' ' ana t h e y then p r o c e e a t o cone up w i t n scme scheme. A c t u a l l y , t n i s i s t h e f i r s t t i n e I have n e a r d t n i s p a r t i c u l a r one, whereas I nave CoTie a c r o s s m e O t h e r s many t i m e s , so D r . Bray should be c o n g r a t u l a t e d on a new scheme f o r s o l v i n g Morse code. L e t me a s s u r e him s a t i t w i l l n o t w o r ~ , we have ~ r i e dit. , i & r s e code m a n u a i i y r e c e i v e d I s n o t t h a t s i n p l e . Tne c o n t e x t n e c e s s a r y a p p a r e n t l y e x t e n d s a t l e a s t 10 l e t t e r s on e i t h e r s i d e - n o t J d s t one s i d e , both s i d e s . I n f a c t , t h e w2s e x a c t l y L Y A C 1. - e a l cyescion a b o u t d o i n s t h i s - N ~ JI chose tlorse code had h a d t h i s k i n d o f i n t e r a c t i o n s ,l eacn o t n e r . YOL. cannot do i t by th l o o k i n & a t b i n a r y f u n c t i o n s o f t h e d u r a t i o n s , t h a t i s , p x p r e s i n g durn.tions a s b i n a r y d i g i t s and m e n l o o k i n g on binary f u n c t i o r s of l o t s of them. I f I taKe 10 l e t t e r s on e i t n e r s i d e , each h a s t h r e e marks, vou s h o u l d c o n s i a e r t h e s p a c e s a s w e l l , b a t l e a v i n g t h o s e o u t , t n a t i s t h r e e tidies 20, m i c h I s 60 d u r a t i o n s , and i f we s p e c i f y t h e n t o one p a r t 11- 32, t h a t I s 300 b i t s , and Dinar;, f u n c t i o n s o f 300 b i t s cannot D S p i c k e d a t rdridGm. 30 t n e q u e s t i o n i s t o take some s t e p s i n t n e r i g h t d i r e c t i o n ana tnen n o r e t l e . s t e p s a r e r i g h t enoi-i&h so t h a t you w i l l g e t enough i m p r o v e ~ e n t s , s o t h a t t h e f o l l o w i n g s t e p s w i l l g e t you even r r r e so.
DR. 0.
~ d i t e d e r s i o n s by the autnor *"is not a v i l l a o l e . v

I m a i n t a i n t R 2 t L:-?re I s some n e r i t In s t u d > i n g s e l f - L ~ r o v e n e n tsystems and i f I an goin; t o do t h a t I an going to s t u d y systems where I Know m a t I want, r a t h e r tnan .?ore 3 i f f i c ~ l t problems, nowever a t t r a c t i v e they may be t o natfie:naLiclans. . - l a i h e n a t i c i a n s 11x2 to work on '-insolved problems for the g r e a t e r g l o r y , and pro'slens a l r e a d y s o l v e d l l ~ en e prl::!e nun'oer theorem t a r e l e f t t o graduate s t u d e n t s . I have a s u s p i c i o n m a t John McCarthy might l a t e r s r i n g up some Important p o i n t s a b o u t d e s c r i p t i o n s , and h e r e I s e e my p o i n t a b o u t s o l v i n g u s e f u l pro'::lens, because Morse code I s a u s e f u l problen. Tne ,+nole q u e s t i o n I s a t '.+hat l e v e l you a r e g i n : to d e a l x i t h d e s c r i p t i o n s o f your data. I n most proLleli-is, e s p e c i a l l y I n d e a l i n g w i t h tne amount o f tr,ln;-s which people do, b i n a r y f u n c t i o n s a r e J u s t n o t adequate. For one tnlnr; t , h e r e is too xucn d a t a , and one of the f i r s t q u e s t i o n s you t a l k about I n the condl t l o n e d r e f l e x - remember t h a t c o n d i t i o n a l p r o b a b i l i t y assumes *t h a t you nave a l r e a d y reco/ni.sed m e s t i m u l u s ; i f J-ou know t h a t the s t i m u l u s celongs to one of a s m a l l c l a s s of f u n c t i o n s , I t I s p r e t t y easy, i s a man, c u t mostly i n human problems you do not know mls. Dr. Mackay nade scme v e r y Kind remarm. I would go f u r t h e r baCK t h a n h e a i d , a s he well knows, i n c r e d i t i n g a e i n o n o l o ~ . A l l o f u s have s i n n e d In Aaam, we have e a t e n 01" m e t r e e 0 1 the knowledge of good and e v i l , and the s demonological a l l e g o r i I s a v e r y o l d one, indeed. A f o r n i s remarks about t h e s y n d i c a t e approach, tne e v o l u t i o n of d i v e r s a t l v e o r ~ a n l s a t i o n , I t h i n k t n a t i s an extremely a c c u r a t e and zood p o i n t . One of the t n l n g s a s p e c i e s l e a r n s I s n o t o n l y how t o s u r v i v e but t o have e x a c t l y t h e rljdit v a r i a n t s In U s memters. The reason r:orseshoe c r a b s have n o t changed much i s n o t Deca-ise they a o n ' t adapt; triey can a d a p t , but t h e r e is no v a r i a t i o n , s o tney can o n l y a a a ? t v e r y l i t t l e , end t n e y a r e a s o u t a s good a s they can be; whereas m e r e a r e a l l Xlnas of people. I n p e h d ' sna i r - f c m a t l o n ~i-ieory, I r e a l 1 c o n s i d e r t h a t speed i n the c l a s s i c s e n s e of computation I s so completely i r r e l e v a n t to t h i s problem m e numser of b i n a r y o p e r a t i o n s one does a second - t h a t i t does n o t i n t e r e s t me vcirj' nuch though i t i n t e r e s t s c o n p u t e r d e s i g n e r s . I woula r a t h e r H A G to s e e l e t s o f o p e r a t i o n s >v!hicii c o u l d conceivably be aone I n p a r a l l e l , 'oein;; done s e o u e n t i a l l y , because t h i s i s the only machine we nave, snd t 3 t 5 I nope p e o p l e w i l l nave machines whlcn w i l l w 0 r K I n CaI'2llel. so t2a.t I want a machine t o do w i c e a s d i f f i c u l t a problem I merely c u l l d m i c e a s D I Ga machine, i n s t e a d o f l c t t i n e , one macnine worA twice a s long. Dr. Strachey a s ~ e d ' x - ~ t e s u l t s , ana they a r e roughly a s I have a r i n d i c a t e d . Irprovement dces t a k e p l a c e ; the n i l l - c l i m b i n g does N C ~ K . The s u t - d e m n s e l e c t i o n I n tr.e prozramnes t n a t we nave d i d n o t do what we hoped, s u t tne %c-clexcns \\'ere e f f e c t i v e l y thrown our, ana tile ones which .';ere k e p t ; e r e 1ar;"ely c o n c e n t r a t e d around t h o s e aerions which r e l a t e t h o s e
r-