Constructing Skilled Images

Virpi Kalakoski

Academic dissertation To be presented, with the permission of the Faculty of Behavioural Sciences of the University of Helsinki, for public examination in lecture room XII, University main building, on 3 November 2006, at 12 noon

Helsinki 2006

Supervised by Professor Pertti Saariluoma Department of Computer Science and Information Systems University of Jyvskyl, Finland Adjunct Professor, Docent Elisabet Service Dalhouse University, Halifax, Canada Department of Psychology, University of Helsinki, Finland Professor Kimmo Alho Department of Psychology University of Helsinki, Finland

Reviewed by Professor Cesare Cornoldi Department of Psychology University of Padova, Italy Professor John T.E. Richardson Institute of Educational Technology The Open University, Milton Keynes, UK

ISSN 0781-8254 ISBN 952-10-3409-2 (pbk.) ISBN 952-10-3410-6 (PDF) (http://ethesis.helsinki.fi) Layout and cover design: Sebastian Therman Yliopistopaino Helsinki 2006

If memory and perception are the two key branches of cognitive psychology, the study of imagery stands precisely at their intersection (Neisser, 1972, p. 233)

Table of Contents
Abstract .......................................................................................................................... 7 Tiivistelm...................................................................................................................... 8 Acknowledgements....................................................................................................... 9 List of original publications .......................................................................................11 1. Introduction.............................................................................................................13 2. Working memory as the seat of representation construction ...........................15 2.1. Modality of working memory ............................................................................15 2.2. The capacity of working memory ......................................................................17 3. The nature of mental imagery ...............................................................................19 3.1. The perceptual properties of mental images.....................................................19 3.2. Mental images in memory ..................................................................................20 3.3. Skilled imagery .....................................................................................................22 4. What mechanisms could underlie expert imagery?...........................................24 4.1. Chunking...............................................................................................................25 4.1.1. The template theory.......................................................................................26 4.2. Long-term working memory ..............................................................................27 4.3. Conceptual knowledge ........................................................................................29 4.4. The constraint attunement hypothesis..............................................................31 5. Incremental Construction of Mental Images ......................................................33 6. The aims of the study..............................................................................................36 6.1. Expertise domains investigated in the study ....................................................36 6.1.1. Chess: Studies I and II ...................................................................................37 6.1.2. Taxi driving: Study III ...................................................................................39 6.1.3. Music: Studies IV and V................................................................................40 7. Method .....................................................................................................................42 7.1. Participants ...........................................................................................................43 7.1.1. Chess players ..................................................................................................43 7.1.2. Taxi drivers .....................................................................................................44 7.1.3. Musicians ........................................................................................................44 7.2. Stimuli and procedure .........................................................................................45 7.2.1. Chess games and positions ...........................................................................47 7.2.2. Lists of street names.......................................................................................51 7.2.3. Musical melodies............................................................................................52 8. Summary of the results...........................................................................................54 8.1. The effect of skill level .........................................................................................54 8.2. Variables affecting the construction of skilled images ....................................59 8.2.1. The surface features of the material.............................................................60 8.2.2. The structure of the stimuli ..........................................................................61

9. Discussion ................................................................................................................65 9.1. Skill level affects the incremental construction of mental images .................65 9.1.1. Experts attune to task relevant constraints.................................................67 9.2. Perceptual chunking has a minor role in imagery construction....................69 9.3. Conceptual chunking underlies skilled imagery ..............................................71 9.3.1. Mental images are used as retrieval structures...........................................72 9.4. Skilled images are constructed in working memory........................................73 9.5. No end to the imagery debate.............................................................................76 9.6. Future issues for research on expert imagery ...................................................80 10. Constructing skilled images.................................................................................84 References.....................................................................................................................85 Original Publications

Constructing skilled images

Abstract When experts construct mental images, they do not rely only on perceptual features; they also access domain-specific knowledge and skills in long-term memory, which enables them to exceed the capacity limitations of the shortterm working memory system. The central question of the present dissertation was whether the facilitating effect of long-term memory knowledge on working memory imagery tasks is primarily based on perceptual chunking or whether it relies on higher-level conceptual knowledge. Three domains of expertise were studied: chess, music, and taxi driving. The effects of skill level, stimulus surface features, and the stimulus structure on incremental construction of mental images were investigated. A method was developed to capture the chunking mechanisms that experts use in constructing images: chess pieces, street names, and visual notes were presented in a piecemeal fashion for later recall. Over 150 experts and nonexperts participated in a total of 13 experiments, as reported in five publications. The results showed skill effects in all of the studied domains when experts performed memory and problem solving tasks that required mental imagery. Furthermore, only experts construction of mental images benefited from meaningful stimuli. Manipulation of the stimulus surface features, such as replacing chess pieces with dots, did not significantly affect experts performance in the imagery tasks. In contrast, the structure of the stimuli had a significant effect on experts performance in every task domain. For example, taxi drivers recalled more street names from lists that formed a spatially continuous route than from alphabetically organised lists. The results suggest that the mechanisms of conceptual chunking rather than automatic perceptual pattern matching underlie expert performance, even though the tasks of the present studies required perception-like mental representations. The results show that experts are able to construct skilled images that surpass working memory capacity, and that their images are conceptually organised and interpreted rather than merely depictive. Key words: expertise, mental imagery, working memory, chunking, chess, music, taxi driving

Eksperttien mielikuvien muotoutuminen

Tiivistelm Mielikuvat ovat muistiedustuksia, jotka sisltvt havaintotietoa. Tss tutkimuksessa tarkasteltiin eksperttien mielikuvien rakentumista tymuistissa, joka on tmnhetkist tietoa yllpitv rajoitettu muistijrjestelm. Tutkimuskysymyksen oli, onko aiemmin opitun tiedon hydyntminen tymuistia kuormittavissa mielikuvatehtviss automaattista hahmojen tunnistamista vai onko kyse muistikuvien muotoutumisesta aiemmin opitun tiedon avulla. Taitotason, rsykkeiden havaintopiirteiden ja rsykkeiden rakenteen vaikutusta muisti- ja ongelmaratkaisutehtviss suoriutumiseen tutkittiin shakinpelaajilla, taksinkuljettajilla ja muusikoilla. Tutkimusta varten kehitettiin menetelm, jonka avulla voidaan tutkia muistikuvan muotoutumisen mekanismeja. Koehenkilille esitettiin yksi kerrallaan shakkipelin siirtoja, peliaseman nappuloita, kadun nimi tai visuaalisia nuotteja ja heidn tehtvnn oli muistaa esitetty materiaali kuvittelemalla kokonaisuus, joka siit muodostuu. Vitskirja perustuu viiteen osajulkaisuun, joissa raportoituihin 13 kokeeseen osallistui yli 150 eritasoista henkil. Tulokset osoittivat, ett kaikilla tutkituilla aloilla ekspertit suoriutuivat muita paremmin muisti- ja ongelmanratkaisutehtviss. Jos tehtvss kytetyn materiaalin rakennetta muutettiin siten, ettei se vastannut tavanomaista jsennyst, eksperttien suoriutuminen heikkeni joka tehtvalalla. Esimerkiksi jos taksinkuljettajille esitettiin kadun nimet aakkosjrjestyksess sen sijaan, ett nimet olisi esitetty reitin mukaisessa jrjestyksess, muistaminen heikkeni. Jos rsykkeiden havaintopiirteit muutettiin, eksperttien suoriutuminen ei juuri heikentynyt. Esimerkiksi shakkinappuloiden korvaaminen mustilla pisteill ei juuri vaikuttanut shakkimestarien suoriutumiseen. Tutkimus osoittaa, ett laajoja ja komplekseja nk- ja kuulomielikuvia, jotka ylittvt tymuistin rajoitukset, voidaan muodostaa vain, jos pystytn hydyntmn silmuistitietoa. Eksperttien omaan alaan liittyvn poikkeuksellisen hyvn muistamisen ja ongelmanratkaisun taustalla ei ole vain tuttujen hahmojen automaattinen tunnistaminen, vaan hitaampi ja tietoisempi prosessi, jossa mielikuva muotoutuu tymuistissa aiemmin opitun tiedon avulla. Eksperttien muistikuvat eivt kuitenkaan ole tarkkoja kuvia vaan tulkittuja muistiedustuksia, joissa tehtvn kannalta epolennaiset piirteet ovat sumentuneet ja olennaiset piirteet korostuneet. Avainsanat: kognitiivinen psykologia, eksperttiys, mielikuvat, tymuisti, mieltmisyksikt, shakki, musiikki, taksiautoilu

Acknowledgements
As developing expertise takes more than 10 years, so did constructing this thesis. And what is often quoted in the expertise research, could also be quoted here: No pain no gain. Nevertheless, carrying out this work in the Department of Psychology, University of Helsinki, has been first and foremost a great pleasure and I wish to express my gratitude to my supervisors. I am grateful to Professor Pertti Saariluoma who led me to cognitive psychology. He offered the opportunity to work in his inspiring research group and to share many invaluable discussions. I thank Docent Elisabet Service for her detailed remarks on my manuscripts and for being a great female role model for doing excellent science. I am also grateful to Professor Kimmo Alho for his comments and advice and for being such a great Head of the Department. I thank Professor Christina Krause for taking on the role of Custos; her efficiency and sense of humour have been really reassuring. Dr. Thomas A. Regelski did not only revise the language of my dissertation, but he also gave many useful comments and suggestions, which I gratefully acknowledge. I have worked at the department since I was an undergraduate student, and therefore there are many other people who have had a great impact on my way of thinking. Dr. Pentti Laurinen has been an exceptional teacher, colleague, and friend who births scientific thinking wherever he goes. I am grateful to him for the many stimulating years. I wish to thank Dr. Jussi Saarinen for his warm support and encouragement and for our inspiring discussions, and my colleague and friend Dr. Pauli Brattico for his passionate and enlightening critique of my research. I also wish to thank all my colleagues in the department for the many happy hours together. Special thanks are reserved for Ms. Sini Maury and Ms. Kaisa Kanerva, and for all the others in our Friday seminars on working memory. Outside the psychology department, I thank my academic friends Docent Jari Ehrnrooth and Ms. Marja Etelmki for advice, discussions and support. I am grateful to Professor Robert Logie for agreeing to be my opponent and to Professor Cesare Cornoldi and Professor John T.E. Richardson for reviewing my thesis and giving me many helpful comments and new insights. I am also grateful for the experts without whom this study had not been possible. Special thanks are reserved for the non-experts who persevered through the experimental tasks that were extremely demanding and unpleasant for them. Financial support from the University of Helsinki, Alfred Kordelin foundation, Anna S. Elonen foundation and the Finnish Konkordia association provided me with the opportunity to conduct this research, which is thankfully acknowledged. Finally, I am grateful to my mother Liisa, my brother Mika, and Petteri and his parents Mikki and Aku for all their help that made it possible to have a loving family at the same time that I was carrying out this research. My most heartfelt thanks go to the most important persons in my life: to my children Samuli, Iikka and Kirsikka, who have given me the opportunity to develop my expertise also outside the academic life, and to Basse. Thank you for your comments, help, and patience, and for sharing not only academic interests but also other visions and daily life.

List of original publications

This thesis is based on the following publications:

Saariluoma, P., & Kalakoski, V. (1997). Skilled imagery and long-term working memory. American Journal of Psychology, 110 (2), 177-201 Saariluoma, P., & Kalakoski, V. (1998). Apperception and imagery in blindfold chess. Memory, 6(1), 67-90. Kalakoski, V., & Saariluoma, P. (2001). Taxi drivers' exceptional memory of street names. Memory & Cognition, 29 (4), 634-638. Kalakoski, V. (in press). Effect of skill level on recall of visually presented patterns of musical notes. Scandinavian Journal of Psychology. Kalakoski, V. (2001). Musical imagery and working memory. In R.I. Gody, & H. Jrgensen (Eds.), Musical Imagery (pp. 43-55). Lisse: Swets & Zeitlinger.

II

III

IV

The publications are referred to in the text by their roman numerals.

13

1. Introduction
People can use mental imagery in memory and thinking tasks, such as memorising the code for their bank account by visualising how it is entered on a numeric keyboard, or imagining the city layout in order to describe to someone how to travel a route from where they are to a destination. Mental imagery can also be used in the auditory modality, for example to imagine the bark of a dog or the sound of ones mobile phone. However, few people are able to apply mental imagery in very demanding tasks such as playing several games of chess simultaneously without seeing the board or pieces, or rehearsing symphonies in their minds. In the present studies, the effect of skill level on incremental construction of mental images was investigated in three domains: chess, taxi driving, and music. The main variables under investigation were related to the effect of skill level and to the effects of experimental manipulations. The effect of skill level indicates the role of pre-learned knowledge and skills in constructing of mental images. The effect of stimulus features, such as modality of presentation or the kind of visual symbols used in the task, was studied to clarify the role of perceptual pattern matching in skilled imagery. The effect of stimulus structure is related to what constitutes meaningful patterns in the domain, and it was studied in order to understand how experts use conceptual knowledge in constructing mental images. The studies described here investigated how experts construct mental representations in memory and problem solving tasks, such as chess players simulating the progress of the game in their mind and successfully constructing the positions of the pieces after several moves, or taxi drivers encoding a list of street names by imagining the underlying spatial route through the city, or musicians memorising notated patterns by transforming the notes into an auditory image of the melody. To tease out the mechanisms involved, a method for studying the incremental construction of mental representations was developed. The focus of the present study, then, is on how experts construct mental images in working memory. The nature of mental imagery has been debated over thirty years (Kosslyn, 1975, 1994, 2005; Kosslyn & Pomerantz, 1977; Pylyshyn, 1973, 1981, 2002), which has lead to a remarkable amount of evidence concerning the use of mental images in memory and thinking (for reviews see Denis, 1991; Kosslyn, 1980, 1994; Paivio, 1991; Richardson, 1980). Currently, there is plenty of data and a detailed theory of the perceptual mechanisms underlying visual mental imagery. However, when experts construct mental images, they do not rely only on perceptual features; they also have access to domain-specific knowledge and skills in long-term memory (LTM). There are only a few examples in mental imagery research of how domain-specific knowledge contributes to mental imagery (Hatano & Osawa, 1983; Hatta,

14

Hirose, Ikeda, & Fukuhara, 1989; Hishitani, 1989, 1990; Saariluoma, 1991), and thus the aim of the Studies I-V was to elucidate this process. Secondly, when experts perform tasks requiring mental images, they are able to exceed the capacity limitations of the working memory (WM) system. The research on expert memory has targeted this issue by studying the interaction of LTM with WM and perception for more than forty years (de Groot, 1965, 1966; Ericsson, Patel, & Kintsch, 2000; Gobet, 2000a, 2000b; Gobet & Simon, 1996b; Simon & Gobet, 2000; Vicente, 2000). The mechanisms underlying expert memory are not yet obvious, and theories suggest partly contradictory explanations. The central question for the present study is whether the facilitating influence of LTM knowledge and skills in WM imagery tasks is primarily based on perceptual chunking or rather on higher-level conceptual knowledge (Chase & Ericsson, 1981; Chase & Simon, 1973; Ericsson, Patel, & Kintsch, 2000; Ericsson & Staszewski, 1989; Gobet, 1998; Gobet & Simon, 1996a; Lane & Robertson, 1979). These are two leading explanations so far suggested in the literature. Since the research is in the intersection of several fields of cognitive psychology (e.g., WM, mental imagery, and expert memory), the literature of these research lines will be briefly reviewed from the perspective of expert imagery. Firstly, the modality of WM and its capacity limitations will be discussed. Secondly, several cognitive levels of mental imagery will be introduced. Thirdly, issues involving expert memory, such as perceptual chunking and the role of conceptual knowledge are discussed and the frameworks of long-term working memory, template theory, and the constraint attunement hypothesis will be introduced. Because it is not obvious whether skilled imagery is specific to the few domains so far studied or whether it is a general phenomenon, three areas of expertise were studied: chess, music, and taxi driving. The first main question was whether pre-learned knowledge and skills affect performance in memory and problem solving tasks that require mental imagery. The second question was how experts construct representations; especially, what is the contribution of perceptual features and conceptual knowledge. To summarise, the effects of skill level, the role of stimulus surface features, and the structure of the stimulus on incremental construction of mental images were investigated in 13 experiments, as reported in Studies I-IV. Study V was a review concerning the nature of musical images and how the concept of WM applies to musical imagery.

15

2. Working memory as the seat of representation construction

Working memory refers to the system or mechanism underlying the maintenance of task-relevant information during the performance of complex cognitive tasks, such as language comprehension, reading, visual imagery, and problem solving (Baddeley & Hitch, 1974; Daneman & Carpenter, 1980; Shah & Miyake, 1999). Several different metaphors have been used to characterise the concept, and the research has led to contradictory claims that may reflect differences in emphasis rather than fundamentally incompatible conceptualisations (Shah & Miyake, 1999). The concept of WM is used in this study instead of the concept of short-term memory; the former focuses on complex tasks, like those that experts face by virtue of being experts, whereas the concept of short-term memory (STM) is used for simpler storage and rehearsal of information (Engle, Tuholski, Laughlin, & Conway, 1999). The basic issues of the WM literature are the modality of the WM system and the nature of its capacity limitations.

2.1. Modality of working memory

The most influential WM model, introduced by Baddeley and Hitch (1974), includes a central executive, which is an attentional control device. It cooperates with two slave systems specialised in the maintenance and processing of language and visuo-spatial information, respectively (Baddeley, 1996). The phonological loop and its articulatory rehearsal mechanisms have been proposed as processing not only language but, in some respect, other auditory information as well; for example, musical images (Baddeley & Logie, 1992; Logie & Edworthy, 1986; Reisberg, Wilson, & Smith, 1991; Salam & Baddeley, 1989). The visuo-spatial sketchpad is proposed to activate visual mental imagery, among other functions (Logie, 1995). It has been divided into the visual component, the visual cache, which temporarily stores visual objects that are subject to decay, and a spatial store, the inner scribe that can be used, for example, to rehearse visual information (Logie, 1995). Several studies that apply the secondary task paradigm have shown the different roles of these subsystems in different tasks. The logic of the experiments using secondary tasks is that these impair the performance of primary tasks that involve the same WM resources. For example, Logie, Zucco and Baddeley (1990) studied the WM subcomponents involved in the visual span task and the letter span task. They used simple mental addition of auditorily presented numbers (a phonological task) and a task where

16

numbers were constructed by imagining the filled cells in an imagined matrix (a visuo-spatial task) as secondary tasks. The results showed that the visual span task was significantly disrupted only by the imagining task, whereas the letter span task was only impaired in the mental addition condition (Logie, Zucco, & Baddeley, 1990). Thus, visuo-spatial and verbal working memories are specialised systems that have separate capacities and mechanisms. The issue of the modality of WM is also implied in the research on the role of a specific WM subsystem in expert memory and problem solving. Research shows that the visuo-spatial sketchpad is the seat of digit memory for mental abacus calculators: memorising digits is more disrupted by a concurrent visuo-spatial task than an auditory-verbal task, while the reverse is true for memorising alphabet letters (Hatta, Hirose, Ikeda, & Fukuhara, 1989). There is also strong evidence that the visuo-spatial sketchpad is involved in chess players domain-specific immediate memory and problem solving. Concurrent visuo-spatial secondary tasks interfere with the memorising of chess positions, whereas articulatory tasks have no effect (Robbins et al., 1996; Saariluoma, 1989, 1992c). Furthermore, chess players problem solving (e.g., move selection), is impaired if they concurrently perform a visuo-spatial tapping task or a central executive task of random number generation, whereas articulatory suppression does not have an effect (Robbins et al., 1996). Thus, besides the visuo-spatial sketchpad, also the central executive has a role, especially in problem solving tasks. However, only concurrent secondary tasks disrupt experts performance; if secondary tasks are interpolated between presentation and recall, there is no interference (Charness, 1976; Frey & Adesman, 1976). These results suggest that experts rapidly store the information in LTM where it can not interfere with WM tasks (Saariluoma, 1992c). Thus, WM is the seat of the early encoding of material and of the construction of representations and thinking processes. Some studies also show the involvement of WM when musicians compare tunes or mentally rehearse melodies. Even though these tasks require processing of sensory attributes, the underlying musical memory mechanisms are not related to sensory memory, which operates only with auditory inputs (Baddeley & Logie, 1992). Instead, musical representations can evoked internally in WM (Baddeley & Logie, 1992). Reisberg, Wilson and Smith (1991, p. 72) describe auditory imagery in WM as follows: one rehearses material in working memory by talking to oneself, and then listening to what one has said. They used the term inner voice for active subvocalisation and the term inner ear for the passive acoustic image, and these terms and processes are widely recognised in both music cognition and the training of musicians (see also Baddeley & Lewis, 1981).

17

There are several studies suggesting a similarity of verbal and musical processing (Salam & Baddeley, 1989) and involvement of acoustic imagery and WM in pitch discrimination (Keller, Cowan, & Saults, 1995; Logie & Edworthy, 1986). Furthermore, subvocal rehearsal and WM are involved in short-term retention of melodies (Logie & Edworthy, 1986). The connection between WM and musical imagery is discussed in more detail in Study V, which was a review of this issue.

2.2. The capacity of working memory

The second major issue concerning WM is the nature of its capacity limitations. In his seminal paper, Miller (1956) proposed a magical number of 7 2 that limits our capacity to process information. However, a recent review offers a substantial amount of evidence for WM capacity of about four cognitive units, or chunks (Cowan, 2001). Research of knowledge rich domains has suggested even smaller capacities of STM: about two or three chunks in visual or semantic memory for Chinese words and idioms (Zhang & Simon, 1985), and only about two chunks consisting of up to 15 pieces each in expert memory for chess positions (Gobet & Clarkson, 2004). Imagery studies have suggested that the size of the matrix one can maintain as an image is 3 x 3 if information is not chunked (Attneave & Curlee, 1983). Moreover, WM may be limited not only by the number of chunks it can hold, but also by the decay over time (Baddeley, 1986; Cowan, 1995). Furthermore, the rehearsal systems also have capacity limits, for instance, articulatory rehearsal in the phonological loop has been suggested to be limited to the amount one can rehearse in about two seconds (Baddeley, Thomson, & Buchanan, 1975). The traditional models of WM have not been able to accommodate the evidence that experts exceptional performance in many cognitive tasks seems to require a capacity that surpasses WM limitations. Ericsson and Delaney (1999) approached this issue by defining how the research into WM greatly varies in the complexity of skills and knowledge required in the studied tasks. The basic-capacity approach uses tasks in which the effects of prior knowledge and experience are controlled as well as possible. The complex activities approach employs everyday activities, such as reading or mental arithmetic, and studies them in the laboratory, for example by using the interference paradigm. The third approach concerns expert performance, which seems to reflect maximal adaptation to WM demands. A distinctive example of this approach is the theory of long-term working memory (Ericsson & Kintsch, 1995) that will be introduced later. Recently, the issue of expert memory has also been taken into account in elaborated versions of the traditional WM models. Baddeley (2000) noted

18

that there are a number of phenomena, central to the present study, that could not be captured by the original WM model. For example, prose recall was a problem as the model could not specify how LTM knowledge is used to increase the span for words in meaningful sentences to exceed the capacity of the phonological loop. Thus, the original model was not able to explain either chunking or the processing of complex information in WM. Therefore, a new WM component, the episodic buffer, was introduced, and a general rehearsal mechanism (involved, e.g., in chunking into subcomponents) was proposed (Baddeley, 2000). The episodic buffer is able to integrate phonological and visual, and possibly also other types of information in episodes. It is also thought to be the seat for chunking information in WM based on LTM knowledge. This concept is relatively new and there is not yet much empirical data to specify the mechanisms involved. The WM approach illuminates the process of representation construction. Different tasks seem to rely on different modules of WM, and capacity limitations restrict the process of representation construction unless LTM is involved. Next, research into mental imagery will be briefly reviewed as a second theoretical basis for research into how mental images are constructed and maintained in WM. Incremental construction of mental representations does not rely on WM only; it also relies on the effective cooperation between WM and LTM. This is central to the third theoretical approach, that of expert memory, which will be discussed last.

19

3. The nature of mental imagery

The nature of mental representations has been a major subject of debate in cognitive psychology for more than 30 years (Kosslyn & Pomerantz, 1977; Pylyshyn, 1973), and it is still continuing (see Pylyshyn, 2002 and the comments on the target article). The distinction between propositional, visuo-spatial and linguistic/temporal codes is fundamental in the literature of LTM (J. R. Anderson, 1983; Engelkamp & Zimmer, 1994; Hitch, Brandimonte, & Walker, 1995; Marschark, Richman, Yuille, & Hunt, 1987; Paivio, 1971, 1986, 1991; Richardson, 1980) whereas the distinction between the visuo-spatial and phonological format is an important issue in the theory of WM (Baddeley, 1986). The debate on whether mental imagery differs fundamentally from other mental representations has led to a substantial amount of evidence for the use of mental images in memory and thinking, and to a theory, based on hundreds of behavioural and brain research studies, on how visual imagery and visual perception depend on partly the same cognitive and brain mechanisms (Kosslyn, 1994).

3.1. The perceptual properties of mental images

The main theory of mental imagery conceptualises visual imagery as a set of representations that gives rise to the experience of viewing a stimulus in the absence of appropriate sensory input (Kosslyn, 2005, p. 334). There are also studies on mental imagery in other sense modalities, such as the auditory (Halpern, 1988; Reisberg, Wilson, & Smith, 1991; Zatorre, Halpern, Perry, Meyer, & Evans, 1996) and the olfactory (Bensafi et al., 2003) but there are no coherent theories for these imagery modalities, as yet. There is abundant evidence for the similarity between imagery and perceptual processes. The so-called functional theories take the position that mental imagery is a medium for simulating the perceptual properties of the external world (Finke, 1985). Studies in line with this approach attempt to explain how mental imagery contributes, for example, to the process of comparing one object with another and to the scanning of visual and auditory images (Halpern, 1988; Kosslyn, Ball, & Reiser, 1978). In comparison with functional theories, structural theories of mental imagery stress that there are some structural similarities between the perception of real and representation of imagined objects (Finke, 1985). For example, musical imagery can represent such attributes as timbre, pitch, and tempo in real objects, as well as in auditory images (Baddeley & Logie, 1992; Crowder, 1989; Halpern, 1988; Hubbard & Stoeckig, 1988; Zatorre, Halpern, Perry, Meyer, & Evans, 1996).

20

Interactive models of mental imagery claim that imagery is mediated by the cognitive and neuronal mechanisms involved in perception (Finke, 1985). Several experimental studies have investigated how mental imagery influences ongoing perceptual processes. For example, Farah and Smith (1983) showed that auditory imagery facilitates detection of same-frequency auditory signals. There is also evidence that imagining visual masks facilitates detection of visual targets similarly to visual perception; this is a low-level phenomenon in the visual system (Ishai & Sagi, 1995). Recent brain research has also demonstrated the overlap between the neural substrates of perception and imagery. There is evidence that some imagery tasks involve activation even as low as in the topographically organised visual areas of the cortex that are employed in visual perception (Kosslyn, 2005). The examples described above represent experiments where mental imagery tasks usually follow the presentation of the stimuli that are to be imagined. Therefore, it is natural that there is overlap between perceptual and imagery processes. Imagery does not, however, require the presence or immediately prior presentation of perceptual stimuli: one can construct mental images from memory.

3.2. Mental images in memory

One major aspect of imagery research is the role of mental images as mnemonics, for example in verbal memory tasks (Paivio, 1971, 1986, 1991; Richardson, 1980). Mental imagery enhances recall of concrete words and sentences, whereas it does not improve memory for abstract words (Paivio, 1986). The dual-coding theory explained this phenomenon by assuming two memory codes. When both the logogens of the verbal system and the imagens of the non-verbal system can be used, as is the case with concrete words, memory recall is improved (Paivio, 1986). The multimodal memory theory (Engelkamp & Zimmer, 1994) extends this assumption and proposes that mechanisms underlying the efficacy of imagery mnemonics are based on change in sensory-motor modality, which necessarily leads to activation of the conceptual memory system. This activation underlies better memory recall when, for instance, representations in the verbal sensory-motor system are transformed into representations in a nonverbal sensory-motor system in connection with mental image construction (Engelkamp & Zimmer, 1994). Furthermore, the effectiveness of imagery mnemonics is also related to the fact that imagery enhances organisation of the to-berecalled items (Bower, 1970; Marschark & Surian, 1989, 1992). From the perspective of working memory research, mental images are perception-like representations that are rehearsed and manipulated in a

21

modality-specific WM (Baddeley & Logie, 1992; Logie, 1995). The major theory of visual mental imagery defines how mental images are generated and maintained. The site for generating, maintaining, and manipulating imagery representations is the visual buffer (Kosslyn, 1980, 1994), which roughly corresponds to the visuo-spatial sketchpad in the WM framework (Kosslyn, 1994; Logie, 1995). Thus, as noted in the context of WM, visual imagery representations, as any WM representations, are believed to decay rapidly and to be disrupted by concurrent processing in visuo-spatial working memory. Reisberg et al. (1991) also conceptualised the mechanisms required in auditory imagery. Musical imagery and its relation to WM are discussed more thoroughly in connection with Study V. When multipart images are generated, as in the present study, individual units have to be integrated to form a complete image (Kosslyn, 1994). Kosslyn suggests four types of image generation. One is based on arranging parts of images using categorical spatial representation, and another is based on using coordinates of spatial relations. For example, expert chess players could construct the image of a position by activating a label for that type of a position and its general pattern and integrate details into it, or they could construct an image by activating the coordinates of spatial relations presented in their memory for a specific chunk. Furthermore, images can be constructed from visual memory by activating the pattern activation subsystem, or they can be generated by engaging attention at different locations in the visual buffer (Kosslyn, 1994). For example, chess players could retrieve visuo-spatial chunks from LTM when imagining a chess position, or they could sequentially attend to different parts of their mental chess board, which implies that complex patterns would require more time to construct (Kosslyn, 1994). Visual mental imagery can also be constructed entirely from LTM by thought (Kosslyn, 1994) or from verbal descriptions (Denis & Cocude, 1989, 1997; Denis & Zimmer, 1992) so that the sensory input from the same modality is not available at all. The coherence of the verbal description affects how easily the image can be generated. If locations on an imagined circular map were presented in random order, participants needed more time to study the description before they were able to construct an accurate image (Denis & Cocude, 1992). Visual images also seem to have gestalt properties and the goodness of figure affects how easily they can be constructed (Saariluoma, 1992a; see also Palmer, 1977). Furthermore, in some imagery tasks conceptual knowledge and interpretations have a relevant role. Kosslyn (1994) suggests that imagined objects are interpreted like interpreting objects in perception: the activation in the visual buffer resulting from the generation of an image is processed similarly to input from the eyes. However, studies on the reinterpretations of ambiguous mental images have suggested that some images are conceptual

22

descriptions rather than perceptual depictions and, therefore, people cannot reinterpret patterns in mental images even though they can easily do it when perceiving the same figures (Chambers & Reisberg, 1985). This suggests that all perceptual features are not represented in images, but the interpretation given in the encoding of images is tied to the representation (Pylyshyn, 2002). Therefore, in the tasks where perceptual events have been recognised or categorised as something, imagery is a top-down conceptual process (Intons-Peterson, 1996; Intons-Peterson & McDaniel, 1991; Perrig & Hofer, 1989). However, the imagery ability of an individual affects the reinterpretation of images, and some individuals are also able to discover alternate interpretations in ambiguous mental images (Mast & Kosslyn, 2002). Furthermore, in imagery tasks that require less WM capacity, people can rearrange patterns in their mental images and recognise new patterns (R. E. Anderson & Helstrup, 1993; Finke, Pinker, & Farah, 1989; Finke & Slayton, 1988), as well as subtract successively presented visual stimuli and find interpretations not evident in the successive parts of the image (Brandimonte, Hitch, & Bishop, 1992a, 1992b). The results of imagery research match the findings of WM research. Maintenance of images is limited by how quickly perceptual features fade, how effectively one can refresh the images, and how effectively one can chunk the material (Kosslyn, 1994). Furthermore, the ability to chunk information is evident when experts perform complex imagery tasks that surpass the limitations of WM. This issue has only recently become recognised; the context of mental imagery research has mainly concentrated on tasks where perceptual aspects are critical and where domain-specific knowledge is not involved. There has been relatively little research to date on mental images that are constructed from LTM and require knowledge of the task domain.

3.3. Skilled imagery

The anecdotal evidence for experts utilising mental imagery (such as Binet, 1893/1966) will not be extensively discussed here but this evidence shows how general the phenomenon is. There is also some empirical evidence that skilled people are able to construct complex mental images which require capacity surpassing the limits of WM. In chess, experts visualise a mental chessboard, and are able to recall the spatial location of individual chess pieces with a location cue which refers to a square on a board (Ericsson & Oliver, 1984 as cited in Ericsson & Staszewski, 1989; Saariluoma, 1991). There are digit span experts who use visuo-spatial maintenance based on a mental abacus in order to remember lists of digit (Hatano & Osawa, 1983;

23

Hatta, Hirose, Ikeda, & Fukuhara, 1989; Hishitani, 1989, 1990; Stigler, 1984). Also, the digit span expert SF used a spatial retrieval structure for encoding lists of digits (Chase & Ericsson, 1981), and the experienced waiter JC memorised dinner orders using the spatial location of customers around tables (Ericsson, 1988; Ericsson & Polson, 1988), and There is also evidence for expert auditory imagery. Research into musical imagery has shown that several musical attributes can also be evoked in the absence of any auditory stimulus, in other words, through auditory imagery in STM or WM (Baddeley & Logie, 1992; Halpern, 1988; Hubbard & Stoeckig, 1988; Keller, Cowan, & Saults, 1995; Zatorre, Halpern, Perry, Meyer, & Evans, 1996). Recent brain imaging studies have demonstrated that auditory imagery is evoked when musicians silently read visual musical notation (Brodsky, Henik, Rubinstein, & Zorman, 2003; Schrmann, Raij, Fujiki, & Hari, 2002). Research on expert imagery suggests that skill effects on imagery reflect differences in domain-specific knowledge in LTM, rather than imagery ability per se (Saariluoma, 1991). Thus, in the case of skilled imagery, the most relevant issues are not related to the perceptual properties of mental images but to the means by which experts are able to construct and maintain images in WM, and how LTM contributes to these processes. Previous studies suggest that pre-learned conceptual knowledge enables experts to chunk information and construct images that surpass limits of WM capacity (Saariluoma, 1991).

24

4. What mechanisms could underlie expert imagery?

Experts exceptionality in cognitive tasks requiring domain-specific knowledge has been well demonstrated in several task environments (for reviews see Ericsson & Kintsch, 1995; Ericsson, Patel, & Kintsch, 2000): for example, in the games of chess and othello (Billman & Shaman, 1990; Chase & Simon, 1973; de Groot, 1966), in sports like ball games and figure skating (Allard & Starkes, 1991; Deakin & Allard, 1991), in arts like music and architecture (Halpern & Bower, 1982; Salthouse, Babcock, Skovronek, Mitchell, & Palmon, 1990; Sloboda, 1985), and in sciences such as engineering and physics (Anzai, 1991; Ball, Evans, Dennis, & Ormerod, 1997). The phenomenon of exceptional performance appears at multiple levels of cognition, such as the encoding and categorisation of material, immediate recall, the organisation of LTM knowledge, and short-term and long-term learning (for a review see Gobet, 1998). However, there are only a few examples in this literature where expertise effects emerge in tasks requiring imagery. Thus, the issue of skilled imagery is a little studied phenomenon not only in the research on mental imagery, but also in the cognitive psychology literature on expertise. Experts outperform novices only in tasks that require knowledge in their field of expertise; in other tasks, such as intelligence tests and WM span tasks, cognitive performance is comparable between different skill groups (Charness, 1988). This phenomenon is referred to as domain specificity. Domain specificity also means that evidence of expertise is found only when stimulus material comprises structures that, from the perspective of the rules or typical practices of the domain, are meaningfully organised. For example, the expertise effect disappears nearly totally when chess pieces are distributed randomly on a board (Gobet & Simon, 1996a). The similar effects of meaningfulness of stimuli on expert memory have also been found in other domains, such as with the game of Go, in figure skating, and with music (Deakin & Allard, 1991; Halpern & Bower, 1982; Reitman, 1976). However, recent research has shown that experts are slightly better than novices even with random material (Gobet & Simon, 1996a, 2000; Gobet & Waters, 2003; Halpern & Bower, 1982; Saariluoma, 1989), which implies that they are able to find chunks in any domain-specific material if they have enough time and even with brief presentation times. Domain specificity of expert exceptional memory suggests that experts have normal cognitive limitations related to perceptual encoding, STM and WM capacity, imagery ability, and the rate of encoding into LTM. Thus, expertise is considered to be representative of general human information processing (J. R. Anderson, 1983; Newell & Simon, 1972), and the perceptual and memory advantages of experts are attributable to experience

25

and learning rather than to a general perceptual or memory superiority (Reingold, Charness, Pomplun, & Stampe, 2001). The problem, then, has been to explain how expert memory and imagery are possible despite the insufficient capacity of STM and the slow rate at which information can be stored in LTM (Richman, Staszewski, & Simon, 1995). The general explanation is that limitations of STM and WM systems can be overcome with the efficient use of LTM knowledge and skills (Charness, 1976; Chase & Ericsson, 1982; Chase & Simon, 1973; Ericsson & Kintsch, 1995). This cooperation between STM and LTM is thought to be based on chunking and the use of conceptual knowledge, which are discussed next.

4.1. Chunking
The original chunking theory postulated that chess masters have acquired a large number of patterns in LTM: from 10 000 to 100 000 configurations, estimated from the computer model (MAPP) that implements the chunking theory (Chase & Simon, 1973; Simon & Gilmartin, 1973). These learned cognitive units, or chunks, are commonly defined as a collection of elements having strong associations with one another, but weak associations with elements within other chunks (Gobet et al., 2001, p. 236). A general theoretical interpretation is that experts are able to use these pre-learned patterns to encode stimulus information as chunks of elements instead of individual elements in STM, and thus surpass STM capacity limitations (Charness, 1976; Chase & Ericsson, 1982; Chase & Simon, 1973; Ericsson & Kintsch, 1995). Experts are therefore exceptional in several cognitive tasks that rely on STM processes, such as encoding, classification, immediate memory, and problem solving. Based on the latencies at which chess pieces were placed on the board in a reconstruction task, Chase and Simon (1973) suggested a STM capacity of seven chunks. They also estimated that each chunk consisted of about five pieces. These numbers nicely fit the finding that chess masters are able to recall almost perfectly even the positions of all 32 pieces. Recent research on expert memory and STM, however, suggests that Chase and Simon overestimated the capacity of STM, and underestimated the number of items in a chunk (Cowan, 2001; Gobet & Clarkson, 2004; Zhang & Simon, 1985). The original chunking theory proposed that experts keep the chunks active in STM (Chase & Simon, 1973). The early studies of chunking mechanisms in chess showed, however, that even when STM is obstructed with delayed recall and interference, experts recall of chess positions remains intact (Charness, 1976; Frey & Adesman, 1976). Therefore, the core

26

of expert memory cannot be the rehearsal of chunks in STM. According to the original chunking theory (Chase & Simon, 1973) LTM learning times were slow, which is not compatible with the large amounts of rapidly presented material that experts can memorise (Gobet, 1998). Two divergent theoretical approaches rose from the problems of the original chunking theory, the template theory (Gobet & Simon, 1996b) and the theory of long-term working memory (Ericsson & Kintsch, 1995). The main difference between these approaches is in what they consider to be the main cognitive level at which chunking occurs (Gobet et al., 2001). There are two levels in the chunking process: an automatic direct perception of familiar chunks, and a slower process similar to problem solving (Chase & Simon, 1973). The idea of perceptual chunking had been proposed already by de Groot (1966) who suggested that chess masters encode large complexes instead of isolated pieces, and that they are able to rapidly see them. Whereas perceptual chunking refers to an automatic and continuous process during perception, goal-oriented chunking is a deliberate process under conscious control (Gobet et al., 2001). Goal-oriented chunking emphasises the importance of higher-level conceptual knowledge in addition to low level perceptual recognition. The conceptual view is exemplified especially in the theory of long-term working memory, whereas the template theory emphasises perceptual chunking. These theories are discussed next.

4.1.1. The template theory The template theory, which is based on the original chunking theory of Chase and Simon (1973), and operationalised in EPAM (Elementary Perceiver and Memorizer) is one of the few computational and carefully formulated theories on expert memory. Although it has been used only in chess research, several of its principles have been suggested as applying to other domains. Its main focus is on the level of perceptual chunking, but it also integrates high-level aspects, such as schematic knowledge and planning, with the low-level mechanisms (Gobet, 1998; Gobet & Simon, 1996a). The template theory proposes three main principles underlying expertise: a large database of perceptual chunks, a large knowledge base consisting of production rules and schemas, and coupling of the perceptual chunks to the knowledge base. It also proposes templates as complex data structures and postulates that verbal descriptions may complement visual encoding (Gobet, 1998). The main feature of the theory is a limited-size visual STM consisting of about three chunks; the minds eye, where the pointers to chunks in LTM

27

are placed. The template theory suggests, like the original chunking theory (Chase & Simon, 1973) that the minds eye is a visuo-spatial system and the site of chess players thinking. It is an internal store of the perceptual and relational structures of objects and a store for visuo-spatial mental operations that can also generate new information. The minds eye is subject to decay and interference (Gobet, 1998). The description of the minds eye is compatible with the visual mental imagery system proposed by Kosslyn (1980; 1994), and the theory of visuo-spatial WM (Logie, 1995). The patterns in the minds eye are constructed from external stimuli and from chunks stored in LTM. Access to chunks in LTM occurs by filtering perceived information through a discrimination net. In the case of atypical material, the chunks consist of only one piece, whereas with typical and meaningful material semantic memory is accessed (Gobet, 1998). Information in the minds eye automatically activates chunks stored in LTM, and they trigger potential moves and plans in the production systems (Chase & Simon, 1973). Gobet & Simon (1998) furthermore stress that the pattern-recognition processes are automatic and unconscious and work from both the perceived stimulus and the internal image. For extensively studied material, chunks are developed into templates that are more complex data structures. Whereas chunks are small, fixed perceptual patterns, such as groups of three to four pieces on a board, the templates are higher-level structures, such as schemas or prototypes (Gobet & Simon, 1996a). For example, in chess, a template indicates the locations of stable pieces, or the core pieces, in a certain type of position. Furthermore, there are slots that indicate pieces and squares that are not fixed for the template but may have default values (Gobet & Simon, 1996a). In addition, there are slots for chess openings, plans and moves, and links to other templates (Gobet & Simon, 1996a). The theory proposes that chunks, templates, and productions, and the pointers linking them together are learned during the acquisition of expertise (Gobet, 1998). Furthermore, retrieval structures that associate games or positions with a pre-learned list (for example, of abbreviations referring to chess word champions) can be used for serial encoding of multiple games or positions (Gobet & Simon, 1996b).

4.2. Long-term working memory

The theory of long-term working memory (LTWM) offers a general framework for a great variety of domains, for example, chess, text comprehension, medical expertise, and digit span mnemonics (Ericsson & Kintsch, 1995). It attempts to specify the general cognitive mechanisms of skilled memory across domains, and it is formulated verbally. General

28

theories allow the testing of the ideas in a variety of empirical tasks and domains, and the later implementation of them with any computational architecture (Ericsson & Kintsch, 2000). The main focus of LTWM is at the level where WM and LTM meet. LTWM is an intermediate memory system that enables subjects to retrieve a substantial amount of information from LTM at a speed similar to short-term WM (Chase & Ericsson, 1982; Ericsson & Kintsch, 1995). The LTWM theory, based on the skilled memory theory (Chase & Ericsson, 1981), proposes three general principles for expert memory: encoding of information with cues related to prior knowledge; decreasing encoding and study times with practice; and retrieval structures that are developed to facilitate the encoding of information in LTM and to retrieve it without a lengthy search (Ericsson & Staszewski, 1989). According to the LTWM theory, experts are able to store incoming information rapidly into a retrieval structure (Chase & Ericsson, 1981, 1982; Ericsson & Kintsch, 1995). The retrieval structure is a stable LTM knowledge structure, or schema, where encoded information is associated with retrieval cues. It is supposed that retrieval structures make it possible for the temporal duration of a memory trace in LTWM to exceed the duration of WM. Two cases that have been studied extensively, the individuals SF and DD, excelled in immediate serial memory for digits (Chase & Ericsson, 1981; Richman, Staszewski, & Simon, 1995). They were able to recall lists consisting of over one hundred digits by using treelike hierarchical retrieval structures that preserved the order of the items. They first encoded the presented digits in groups of three or four and then connected these groups to subgroups and the subgroups in their turn into clusters (Chase & Ericsson, 1981; Richman, Staszewski, & Simon, 1995). The digits in a group were associated with each other mostly on the basis of semantic LTM knowledge of typical running times, which was a familiar domain for both of these subjects. Thus, the digits were not only related to the retrieval structure, but also to each other and the patterns and schemas in LTM (Ericsson & Kintsch, 1995). Digit memory experts like SF and mental abacus calculators are able to use the location of the digit in the list as a recall cue, and they are also able to reproduce digit lists backwards as fast as forwards (Chase & Ericsson, 1982; Hatano & Osawa, 1983). In chess, the retrieval structure relates different pieces to each other and to their locations on the 64 squares of the board. The information can also be encoded by elaborating LTM schemas (Ericsson & Charness, 1994; Ericsson & Kintsch, 2000). Thus, chess players can even encode random positions, since they are able to rely on their knowledge and generate transformations and partial matches to familiar patterns if they have sufficient presentation time (Ericsson, Patel, & Kintsch, 2000, p. 585).

29

In the domain of music, the hierarchical retrieval structures for performing a composition are developed in practicing the piece. They are based on the formal structure of the composition (Williamon & Egner, 2004; Williamon & Valentine, 2002). When musicians are sight reading or improvising music they cannot rely on preformed retrieval cues as the music has not been rehearsed before (Williamon & Valentine, 2002). In that case, the previous 'musicianship' knowledge and the particulars of the current context affect performance. Performing from notation relies on a combination of these factors (Williamon & Valentine, 2002). Retrieval structures can be deliberately acquired and are available to consciousness (Ericsson & Kintsch, 1995). It has been claimed that they are only used in domains in which memory improvement is the main task, and it is not obvious how LTWM explains the skill effects in contrived memory tasks where experts are not familiar with using retrieval structures or other mnemonics in their daily activities (Vicente & Wang, 1998). Furthermore, the theory of LTWM has been claimed to be applicable only to such domains as text comprehension and superior digit memory, where order memory, conscious effort to apply a mnemonic, and serial encoding are important; it is less clear to what extent they are able to explain other domains of expertise, such as chess (Gobet, 1998). LTWM has also been criticised for including many unspecified mechanisms and structures, and for being too vague to generate predictions (Gobet, 2000b). However, although verbal theories have weaknesses, they offer a way to search for general cognitive principles and have been able to summarise a large body of empirical data (Ericsson & Kintsch, 2000; Gobet, 2000b). The template theory, on the other hand, even though being well-specified, is applicable in its current form only to a very restricted number of tasks in chess.

4.3. Conceptual knowledge

Knowledge-based frameworks stress that there are important qualitative domain-specific differences in the organisation of knowledge, WM representations, and problem solving strategies (Gobet, 1998). Knowledgebased frameworks account for the role of organisation of knowledge in LTM, levels of processing, and thinking processes. For example, expert computer programmers and physicists seem to represent problems at an abstract level and around fundamental concepts, whereas novices focus more on surface features, dominant objects, and concepts mentioned in the task (Adelson, 1984; Chi, Glaser, & Rees, 1982). In chess, the ability to see individual chess pieces as parts of larger, meaningful units suggests an abstract representation rather than a concrete perceptual pattern (Adelson, 1984). Furthermore, chess players of different

30

skill levels are distinguished more by the ability to perceive and produce connections between chunks rather than the ability to distinguish the chunks as such (Freyhof, Gruber, & Ziegler, 1992). Experts must also orientate to the task in ways that utilise their superior understanding about the domain (Goldin, 1978). Therefore, the depth of processing (Craik & Lockhart, 1972; Lockhart & Craik, 1990) affects the recall of chess positions. Semantic and meaningful orienting tasks, such as choosing a move, and intentional learning tasks facilitate chess players recall and recognition of positions at all skill levels. However, skill level does not affect recall in a non-meaningful orienting task condition, such as piece counting (Goldin, 1978; Lane & Robertson, 1979). Furthermore, tasks that do not encourage semantic processing but allow pattern-matching, such as copying the chess position, do also improve recognition performance but not to the level of semantic orienting tasks (Goldin, 1978). These results show that pattern matching alone does not explain the expertise advantage. There is also other evidence indicating that higher-level knowledge, more abstract than perceptual chunks, underlies expert memory (Cooke, Atlas, Lane, & Berger, 1993). For example, giving a description of the chess position before its actual presentation improves recall (Cooke, Atlas, Lane, & Berger, 1993). Furthermore, when several game positions are recalled, the meaningful units at recall seem to be the whole positions rather than individual chunks (Cooke, Atlas, Lane, & Berger, 1993). Some researchers have proposed that expert performance does not rely on chunks and pattern recognition at all. For example, the SEEK (Search, Evaluation, Knowledge) theory of chess skill put forward by Holding (1985) claims that rich and highly organised knowledge such as prototypes, general principles for chess relationships, common themes, and chess knowledge organised around verbal labels are used rather than chunking. Furthermore, according to this theory, skill is not based on recognising specific patterns but on thinking ahead (forward search) and on the evaluation of move sequences (Holding & Reynolds, 1982). For example, although chess masters and less skilled players differ little in their ability to recall briefly presented random positions, the best moves chosen for the random positions improve as a function of skill level. These results suggest a component in expert memory organisation that does not affect direct recognition, but that emerges when there is enough time for the evaluation of the material. The apperception-restructuring theory of chess memory (Saariluoma, 1992b, 1995) emphasises the important role of information selection in skilled performance. Saariluoma (1995) criticises the views that emphasise the role of capacity-limitations in selection and proposes that conceptual selection of information is more decisive. The capacity limitations of attention have a role in the selection of information from an actual environment: however, when mental representations are constructed, the

31

issue of sensefulness is critical (Saariluoma, 1995, pp. 99-135). The term senseful is introduced as a technical term to refer to sensible wholes and to express that the contents of a representation make sense. It is used instead of the problematic term meaningful. The theory proposes that chess players seeing is not perception as such, but apperception, the conceptual perception or construction of semantic representations (Saariluoma, 1995, p. 102). In apperception, perceptual stimuli and conceptual memory information are assimilated into a senseful representation. These mental spaces can be changed in the process of restructuring, where the contents of representations are reorganised (Saariluoma, 1995, p. 136). When chess players construct representations, they seem to follow simple contentspecific principles that define what is essential (Saariluoma, 1995). The process of apperception is based on these unconscious and implicit principles that separate essential from inessential properties (Saariluoma, 1995).

4.4. The constraint attunement hypothesis

The above summary of approaches provides a psychological explanation for expertise effects on memory by proposing the mechanisms of chunking and cooperation between LTM and STM systems. These approaches have recently been criticised for not providing sufficient explanations for domains other than those in which memory recall is the core of expertise. According to Vicente and Wang (1998), intrinsic memory tasks are found in domains in which memorising is the main feature. For example, for the digit memory expert SF recalling digits is the task he usually performs. In contrast, contrived memory tasks are not parts of normal tasks in other domains, rather improved memory is a by-product of the special skill. For example, the main occupation of chess players is not to recall briefly presented positions but to play the game. The constraint attunement hypothesis (CAH) claims that there is a need to step back and develop a theory of the task environment before the models of cognitive mechanisms can be improved. Instead of describing psychological mechanisms, the CAH takes an ecological approach: the object of theorising is the environment itself rather than its mental representation (Vicente, 2000). The aim of the CAH is to provide a hierarchy of abstractions that describes the environment. By moving up the hierarchy details decrease and the goals of the system are presented. In moving down the hierarchy concrete ways to carry out goals are found (Vicente & Wang, 1998). In chess, for example, the highest level, purpose, is the players goal, such as to win the game. The next level is that of strategies, the plans that a player has adopted to achieve the goal to win. A lower level,

32

tactics, includes effective ways to implement a strategy. At the next level there are the legal paths that can be selected to execute a tactic. The lowest level is the board and the physical configuration of pieces. Thus, the hierarchies are linked by a means-end relation. Memory for higher levels provides help in constructing representations for lower levels; for example, a strategy or tactic helps to specify the position of individual chess pieces. The CAH supposes that, in general, the level of recall is a function of the number of goal-relevant constraints that experts can take advantage of to structure the stimuli (Vicente & Wang, 1998). The more constraints that are available, the more meaningful and natural are stimuli. Thus, the meaningfulness of stimuli is defined ad hoc from the abstraction hierarchy of the environment. The CAH predicts that skill effects are greater with stimuli that have more constraints. Frey and Adesman (1976) found that recall improves when the presentation condition changes from random to meaningful, and to move-by-move presentation. These findings are interpreted as reflecting a greater number of constraints, such as strategies and tactics, which are available in the move-by-move condition (Vicente & Wang, 1998). The improvement of recall with a semantic orienting task, as compared with formal orientating (e.g., Lane & Robertson, 1979), is claimed to be consistent with the importance of attuning to goal-relevant constraints of the environment (Vicente & Wang, 1998). Thus, the CAH is related to theories of skill that emphasise conceptual factors: the SEEK theory (Holding, 1992) and the apperception-restructuring view (Saariluoma, 1992b, 1995). Attunement to constraints is claimed to be a prerequisite for information search that is central in both of these theories (Vicente & Wang, 1998).

33

5. Incremental Construction of Mental Images

The preceding review makes it obvious that there is no consistent view of how experts construct mental imagery representations. The imagery processes described above do not provide a detailed description of the mechanisms underlying the task used in Studies I-IV: that is, when experts incrementally construct a mental representation. The tasks that experts are able to perform are complex and require capacity surpassing WM limits. Therefore, associative memory and conceptual knowledge may have a greater role than in the simpler imagery tasks studied earlier. The Studies IV-V also investigated the domain of music, where auditory imagery has a greater role than visual imagery. There is presently no clear picture of the general principles that underlie imagery in different sense modalities in the complex images that experts are able to construct. There are, however, striking similarities between the main questions of imagery research and research on expert memory: in both fields the roles of perceptual and conceptual knowledge are debated. In the imagery literature, the disagreement about whether perceptual properties are the core of mental imagery or whether interpretations are tied to the imagery representations is an important issue. In expert memory research, the dichotomy between perceptual chunking as the main mechanism underlying expert memory and the crucial role of higher-order knowledge and conceptual chunking is also essential. Furthermore, several theories of expert memory imply that mental imagery has an important role when experts construct representations. The chunking and template theories propose a visuo-spatial system, a minds eye, as the foundation of chess players thinking. The LTWM theory suggests spatial retrieval structures for several task environments. Thus, although the imagery research and the expert memory research represent different traditions, the dichotomy of perceptual and conceptual knowledge is found in both. What, then, could be the mechanisms underlying experts mental imagery? The importance of perceptual properties is evident in the concept of minds eye introduced in expert memory research. The patterns in the minds eye are constructed from external stimuli and from LTM. These descriptions are in line with the suggestions of imagery research. Kosslyns (1994) theory on visual mental imagery, and research findings on visuospatial WM (Logie, 1995) further suggest several processes and components that are required in the generation, maintenance, and manipulation of images. Furthermore, the chunking and template theories of expert memory stress that the pattern-recognition processes are automatic and unconscious, and operate on both the perceived stimulus and the internal image (Campitelli & Gobet, 2005; Gobet & Simon, 1996a). These assumptions lead

34

to the hypothesis that the main mechanism underlying skilled imagery is perceptual chunking. However, expert imagery is a very complex phenomenon and, therefore, it is possible that the perceptual imagery processes that have been demonstrated in knowledge-poor tasks are not sufficient in explaining expert imagery. There is evidence that, at least in some cases, interpretation is tied to mental imagery and that conceptual knowledge plays a major role (Chambers & Reisberg, 1992; Reisberg, 1996; Reisberg, Smith, Baxter, & Sonenshine, 1989). There is also preliminary evidence for the claim that the level of LTM knowledge and skills and the ability to overcome WM limitations contribute to how experts construct mental images (Saariluoma, 1991). Furthermore, theories of expert memory stressing the role of higherlevel conceptual knowledge suggest that constructing skilled images can be a slow problem solving process rather than fast automatic pattern recognition. If this is the case, mental imagery could offer a special kind of system for encoding and retrieving information deliberately and consciously. These assumptions lead to the hypothesis that skilled imagery is not solely based on perceptual chunking and that conceptual chunking also plays an important role. One reason for the lack of consensus among mental imagery researchers and among expert memory researchers is the great variety of empirical methods used to study the basic issues in these research fields. In imagery research, the main problem in developing a consensus between theorists is that the current views on imagery are based on a mixture of behavioural and brain research studies, as well as on introspective and intuitive notions (Pylyshyn, 2002). Therefore, the empirical results are not strictly comparable. For example, in some studies the mental images result from an immediately prior visual stimulus but in some cases the images are constructed from LTM. However, results concerning sensory images are often interpreted as being evidence for the perceptual properties of mental images although it is unclear whether the images following sensory stimuli and those constructed from memory actually reflect the same phenomenon. In the expert memory research, empirical methods vary from brief-exposure techniques customarily employed by researchers of perceptual chunking, to memory tasks, verbal protocols, and long-range learning tasks used by other researchers (Gobet, 1998). In this situation, the empirical data on expert memory are disconnected and difficult to compare between several domains, and even within a single domain. In the present research, a method was developed for use across a variety of domains. This method can provide empirical data relevant to the further development of the current theories on mental imagery and expert memory. The research method was developed from an application of blindfold chess (Saariluoma, 1991). In blindfold chess, players do not see the chess board

35

but have to imagine the moves of the pieces that are described to them verbally. In the present studies, representations were constructed from successively presented stimuli for problem solving or immediate recall. Thus, the participants did not see whole stimuli patterns, such as all the pieces of a chess position or every street name of the list or all the notes of the pattern. This method makes it possible to tease out the process of representation construction, since experts are not familiar with this kind of task; thus the process cannot be fully automatic, and therefore its mechanisms can be caught. Since an equivalent experimental method was applied to every domain investigated in the present studies, it was possible to compare results from the different domains. This approach addressed general cognitive principles that underlie skilled imagery in several domains. In this sense it resembles the approaches of LTWM and the constraint attunement hypothesis. The domains of chess, taxi driving, and music were studied. In these domains memorising is not an intrinsic task, but a contrived task, an issue considered important in the template theory and the constraint attunement hypothesis. That is, although chess players, taxi drivers, and musicians have to memorise games, routes, and melodies, the defining feature of these domains of expertise is not memorising the stimuli; exceptional memory and imagery are by-products of expertise. At the same time, the present studies concentrated on a specific process in expert imagery: the incremental construction of mental representations. Therefore, the approach also resembles that of template theory, where the focus is on a specific task and the aim is to understand specific mechanisms. Thus it was possible to clarify both the general aspects of expert memory and imagery across three domains, and the specific mechanisms underlying the process of incremental construction of mental imagery. In the representation construction task used in the present study, the role of WM is crucial: a person has to maintain the presented items in WM in order to construct a global representation and to connect individual items into chunks. This process is subject to decay and interference. The main question is how pre-learned knowledge and skills that experts have acquired contributes to this process and how experts are able to overcome the limitations of WM. What is at issue is the role of automatic pattern matching in image construction, and whether experts are also able to use higher-level knowledge and conceptual chunking.

36

6. The aims of the study

The aim of the present research was to develop a method for studying the encoding of pieces of information into meaningful chunks and to use this method to provide further information about the effect of skill level in the process of constructing mental representations. The first question was, Does pre-learned knowledge facilitate representation construction even when the presentation of stimuli is piecemeal? That is, are the typical expertise effects, such as better recall levels by experts, also found if the stimulus is not presented as a whole and, thus, when the representation has to be mentally constructed? If mapping of whole patterns from LTM is not required to produce an expertise advantage, then the results will show the effect of skill level on recall and problem solving when the present method is used. The second question was, Do surface features of the stimulus material have a role in representation construction? That is, do the presentation mode and the specific symbols that are used to communicate the information influence recall and problem solving? For example, with one type of notation, such as using letters rather than visual notes or algebraic notation instead of typical chess pieces, the relation between symbols and domain-specific material is less direct than what the experts normally meet in their daily activities. If expertise advantage is based on perceptual visual chunking, then experts will perform better in the familiar presentation condition than in the condition where typical visual cues are not available. In contrast, if mental representation construction depends on factors other than the visual features of the stimuli, the presentation mode should not affect performance. The third question was, Does the structure of the stimuli and orientation to the task affect recall and problem solving? That is, do transformations of stimuli and orientation instructions affect expert performance? For example, moving one single chess piece to another position on a board can affect the continuation of the game, and organising streets in a list according to their spatial organisation on a map rather than in alphabetical order is related to the number of task relevant constraints available. If higher-level conceptual knowledge has a role when experts construct mental images, then the structure of stimuli and orientation instructions influence recall and problem solving and some parts of images are recalled better than others.

6.1. Expertise domains investigated in the study

The role of chunking and conceptual knowledge underlying expert performance was investigated in all three domains of this study: chess, taxi

37

driving, and music. The domains, however, differ in several aspects. Whereas chess and taxi driving are visuo-spatial domains, music is principally auditory in nature. While chess players operate in a small-scale visual environment, taxi drivers are experts in dealing with a large visuospatial surrounding. From another point of view, in all of these domains the serial presentation of material is common. In chess, the games proceed in move sequences, although the positions are in essence visuo-spatial information. In taxi driving, the driving routes unfold as a series of visuospatial information, and in music the notes are arranged in temporal sequences. Thus, the domains overlap in some properties and differ in other aspects, which is a normal case in the research on expertise. The domains under study also differ in the amount of research that has been conducted. A mechanical search, without editing the results, in the PsychArticles and PsychInfo databases was conducted in May 2006. With the key words chess and memory, 143 references were found from 19072005. In contrast, for taxi and memory, there were only 11 references from 1950-2005, whereas music and memory yielded 994 references from 19032006. These findings demonstrate the importance of chess research in cognitive science, and the relevance of music research in understanding broader issues such as the processing of language and the development of music cognition throughout the life span.

6.1.1. Chess: Studies I and II Chess offers an ideal task domain for the study of expert performance and has therefore often been called the Drosophila, or fruit fly, of cognitive scientists (Reingold, Charness, Pomplun, & Stampe, 2001). There are several ways in which studying chess is advantageous: for instance chess is a complex but still mathematically formalised task which has external and ecological validity, and offers a precise scale for quantifying the level of expertise (Charness, 1992). Chess is therefore a well-defined task environment and has had a great impact on cognitive science (Charness, 1992). The game of chess is played by moving 16 white and 16 black pieces, following the rules of the game, on a chessboard with 64 squares. The players must be able to discriminate between six kinds of pieces. The visuospatial organisation of the pieces on the board changes after each move. Chess semantics includes attacks and defences, and subtle moves that are aimed to improve the favourability of the position (Goldin, 1978). Tactics and strategies add several levels of meaning to the game. The progress of a game can be split into the opening, the middle and the end game, and expert

38

players are familiar with variations of each. Typical variations and situations have also been named, such as the Queens gambit or a rook ending. Cognitive psychologists have studied memory and thinking in chess using a variety of methods, and the domain specificity of expert chess memory is well demonstrated (for reviews see e.g., Gobet, 1998; Saariluoma, 1984, 1995). As noted above, despite a large body of research, there are major discrepancies in the interpretation of the mechanisms underlying chess memory. The research on blindfold chess is especially central to the present study. In blindfold chess, the players are not able to see the chess board. Moves are communicated using standard chess notation indicating which piece moves to which square on the board; for example, Kh4 indicates moving the Knight from its current location to the h4 square on the board. The cognitive processes involved with blindfold chess have been systematically studied for only about twenty years by Saariluoma, although there is some earlier anecdotal evidence (for a review see Campitelli & Gobet, 2005; Saariluoma, 1991). Previous research has shown that domain specificity is also demonstrated in blindfold chess (Saariluoma, 1995). Moreover, the results suggest that the processing of games and positions occurs in visuo-spatial WM, which is crucial in the early encoding of games and positions as well as in problem solving (Robbins et al., 1996; Saariluoma, 1992c). The representations of positions and games stored in LTM are not sensitive to distracting tasks interfering with visuo-spatial WM. Moreover, it is not ability for imagery per se, but LTM knowledge that underlies the effect of skill level on blindfold performance (Waters, Gobet, & Leyden, 2002). The present study continues the research approach started by Saariluoma (1984; Saariluoma, 1991), and applies an analogous method to the other two domains, taxi driving and music. In the present thesis, the following more specific issues were investigated in order to clarify the mechanisms underlying skilled imagery in chess. Study I. The role of stimulus features and perceptual pattern matching in memory for chess games were addressed. The specific questions were: Does the number of visual attributes (colour and kind) affect the recall of chess games? (Experiment 1) Are move patterns in chess encoded absolutely; that is, does the spatial transposition of the board affect recall? (Experiment 2) Does presentation modality (auditory vs. visual) affect recall? (Experiment 3) Are experts faster in encoding chess-specific information than mediumlevel players? (Experiment 4)

39

Study II. Attunement to task constraints and whether all parts of a chess position are equally represented were studied with memory and problem solving tasks following the blindfolded presentation of positions. The specific questions were: Are transformations of one single piece of a chess position recognised more easily if they affect the functional figure than when the transposed piece is situated in the functional periphery? (Experiment 1) Does the orienting task (move-selection vs. counting) affect recognition of minimally transposed positions? (Experiment 2) Does randomisation of the background affect problem solving and recall of positions? (Experiment 3) Does a visuo-spatial secondary task interfere with chess players problem solving and recall? (Experiment 4) Does the presentation format (fully visible, blindfolded visual, blindfolded auditory) affect chess players problem solving and recall? (Experiment 5)

6.1.2. Taxi driving: Study III Taxi driving is a skill that involves a large-scale spatial environment: a whole city with its streets, landmarks, and main driving routes. The skill also requires driving ability and knowledge of traffic rules. In Helsinki, to be qualified as a taxi driver, a person must take a course and pass examinations requiring knowledge of streets and driving routes. There are only a few studies on the cognition of taxi drivers (Chase, 1983; Maguire, Frackowiak, & Frith, 1997; Pruch, Giraudo, & Grling, 1989). Previous research has shown that experts can name more streets than novices, and are far more skilled at generating driving routes (Chase, 1983). Furthermore, locations in the city seem to be hierarchically organised in experts memories. Experts can also efficiently use perceptual knowledge to trigger representations of known routes associated with visual cues (Chase, 1983). A brain imaging study also indicates that the hippocampus, a brain structure related to the processing of spatial information, is larger in taxi drivers than others (Maguire, Frackowiak, & Frith, 1997). Although skill effects have been demonstrated in the domain of taxi driving, it is not obvious whether taxi drivers can facilitate immediate verbal memory recall by using their pre-learned visuo-spatial knowledge. The main task of taxi drivers is to generate the best routes between locations in the

40

city; it does not involve immediate serial recall of street names. However, if they also demonstrated an exceptional immediate serial memory, it would be a new example of enhanced memory recall as a by-product of a specific skill. Study III. The general question was whether spatial retrieval structures can be used in a verbal memory task. The specific questions were: Does knowledge concerning a large-scale visuo-spatial environment affect immediate serial verbal recall of street names? (Experiments 1 and 2) Does the level of task-relevant spatial information available in the lists of street names (lists organised into spatially continuous routes, noncontinuous routes or in random order) affect recall? (Experiments 1 and 2) Are other kinds of mnemonics (street names in a list belonging to the same category or arranged in alphabetical order) as efficient as spatial organisation? (Experiment 2)

6.1.3. Music: Studies IV and V Although tonal Western music, like language, is a highly structured system, a thorough formal description of its grammar has not been achieved (Tillmann, Bharucha, & Bigand, 2000). The regularities of tonal music are acquired not only in music education, but also through implicit learning in everyday life. Therefore, knowledge of complex regularities of melodic, metrical, and rhythmic structures is partly explicit for musicians and tacit for both musicians and non-musicians (Tillmann, Bharucha, & Bigand, 2000). As in several other task domains, the typical effects of skill level and meaningfulness of the stimulus material can also be found in music. For example, skilled musicians can perceive and memorise musical melodies and rhythm patterns better than non-skilled subjects, and musicians are able to recall musically well-organised structures better than random structures (Charness, 1988; Clifton, 1986; Halpern & Bower, 1982; Meinz & Salthouse, 1998; Roberts, 1986; Sloboda, 1976, 1985). Moreover, there is direct evidence based on eye fixations (Waters, Underwood, & Findlay, 1997) that indicates that musical experts use larger chunks than novices when comparing two visually presented note patterns, and that experts process these units with fewer fixations and in less viewing time. Moreover, several studies suggest that music is processed in WM as auditory images (Baddeley & Logie, 1992; Logie & Edworthy, 1986; Reisberg, Wilson, & Smith, 1991; Salam & Baddeley, 1989).

41

Study IV. The general issue was whether musicians use auditory imagery when constructing representations from visually presented patterns of musical notes. In addition, the role of stimulus features and perceptual pattern matching was addressed. The specific questions were: Are only musicians better able to recall real but unfamiliar melodies in comparison to recalling mirror images of melodies? (Pilot experiments, Experiment 1) Does the presentation format (notation vs. letter names for notes) affect the recall of melodies and the distribution of recall errors? (Experiment 2) Study V. The question to be answered by this review was: How is musical imagery processed in working memory?

42

7. Method
Although the nature of mental imagery is one of the main issues in studies of cognition, such studies have seldom employed tasks that tease out the process by which representations are constructed. This process was studied here, using a new task in which individual stimuli needed to be combined to construct a unitary representation. The stimuli were presented serially so that the pattern could not be perceived as a whole, and the fragments of information had to be actively maintained in WM, until there was a larger body of information that could be connected with pre-learned knowledge or chunked into units. Thus, the capacity to maintain such information in WM before a familiar pattern was discernible was critical for successful chunk construction, distinguishing the task from more typical expert tasks in which the instant mapping of LTM patterns on presented stimuli is possible. Consequently, this kind of method captures the cooperation between WM and LTM that is assumed to underlie skilled imagery and that is required when the fragments of information are united into a meaningful chunk or multiple chunks. The successive presentation of stimuli makes it possible to study the incremental process of imagery construction for three reasons. Firstly, the patterns have to be constructed in the mind from the visually or auditorily presented items. Secondly, the patterns are not presented in their entirety. Thus, the construction of representation has to be accomplished without the help of an immediately available visual or auditory gestalt. Finally, the patterns are domain-specific configurations that consist of a total number of items that exceed the capacity of WM. Therefore, the present task is not a pure WM task but requires, instead, the use of pre-learned knowledge and skills for the memorisation of the presented items. Previous research applying an analogous method concentrated on blindfolded chess (Saariluoma, 1989, 1991). These studies showed that the construction of meaningful units is affected by skill level and mediated by mental imagery, at least in the visual domain (see also Hatta, Hirose, Ikeda, & Fukuhara, 1989; Hishitani, 1989). Also, in the domains of taxi driving and music, a theoretically important question is whether the effective chunking of incoming information into meaningful units can be done even if complete patterns are not immediately accessible to encoding processes but can only be assembled in WM. In all the studied domains, common methodological principles were followed. Participants were experts in their domain, and in about half of the experiments they were compared to less skilled or novice participants. Stimuli were task-specific material, and details of presentation or organisation were varied.

43

7.1. Participants
The participants were experts in one of the studied fields, chess, taxi driving, or music. The control groups consisted of participants with some or no knowledge of the task environment. It was hypothesised that if an effect of skill level was found, it would reflect the role of domain-specific knowledge and skills in mental imagery. Only participants of age 18-40 years were chosen for the experiments, since in this age group, basic cognitive skills are fully developed, but agerelated cognitive decline is still minor (Verhaegen & Salthouse, 1997). The study was conducted in Helsinki, with participants recruited from the greater Helsinki urban area. Both the expert and novice chess players were recruited from chess clubs, the taxi drivers through their union, and the musicians through announcements for volunteers in various music institutes. The control groups consisted mainly of students at the University of Helsinki. The experts were paid for their participation, and the control groups participated for a course credit or a small payment. Overall, the 13 experiments and two pilot experiments of the present study included 165 testing sessions, involving over 150 participants. The detailed descriptions of participants are found in the context of the original experiments. The number of participants in each experiment is presented in Table 1.

7.1.1. Chess players The overall number of participants in the chess experiments was 69. There were 27 chess players in Study I and 37 chess players in Study II 1 . The ELO chess expertise rating system (Elo, 1978) was used to define the Finnish expertise rating (SELO) of the participants. The ELO rating system has a mean of 1500 and a standard deviation of 200 and it is based on the chess players wins, loses, and draws in chess competitions. The size of the change of ones rating in a competitive game also depends on the rating of ones opponent. It is a reliable measure of the skill level in chess, and the difference in the ratings of two players expresses the probability of winning (for details, see Elo, 1978; Holding, 1985, pp. 9-12).

In Study I there were 32 testing sessions, since five players took part in two experiments. Information about whether some players participated in more than one experiment in Study II or whether some of the players participated in both Study I and Study II is not available. Therefore, the number of individual chess players is a little smaller than the overall number of participants in the testing sessions.

44

The participants for Studies I and II were chosen for study on the basis of their SELO rating. In experiments with skill level manipulated as an independent variable, the compared groups represented Grand Masters with SELO ratings of 2200 to 2400 and Medium-level players with SELO ratings of 1600 to 1850. If the effect of skill level was not investigated in the experiment, the range of SELO ratings varied between 1600 and 2400 points, depending on the experiment.

7.1.2. Taxi drivers This expert group consisted of eight Helsinki taxi drivers. In Helsinki, to qualify for a taxi driving license a person must take a course and pass examinations concerning knowledge of the streets and driving routes. All participants had this qualification and 3-7 years of experience of full-time taxi driving in the Helsinki city area. The control group consisted of eight students from the University of Helsinki. The same participants took part in both experiments.

7.1.3. Musicians There were 20 musicians and 20 control participants in the pilot experiments of Study IV. In Experiments 1 and 2, the expert groups consisted of a total of 20 musicians who had actively played one or several musical instruments for 10-28 years. They were professional musicians or music educators, or they were studying towards a professional degree in music. They had also completed several music theory and performance modules included in their formal education. All musicians were experts in classical music and were highly experienced in, for instance, singing or playing melodies from notes and notating melodies after hearing them once or a few times. The groups of musically un-trained subjects in experiments 1 and 2 consisted of 20 students of psychology. Some of them had played a musical instrument as a child for less than five years. None had practised any musical instrument for several years or were able to fluently read musical notation. In contrast to musical experts, non-musicians do not receive memory support from pre-learned auditory or music-related knowledge when they are processing visual notes. They are, therefore, presumably, confined to using only the visual modality or verbal rehearsal in the integration of visually presented individual notes into a whole. None of the musicians or the non-musicians had participated in the pilot experiments.

45

Table 1. Participants and stimulus material Experiment Study I 1 2 3 4 Study II 1 2 3 4 5 Study III 1 2 Study IV Prel. exp. 1 2 20 musicians 20 non-musicians 8 musicians 8 non-musicians 12 musicians 12 non-musicians Musical melodies consisting of 11-16 quarter and half notes Musical melodies consisting of 10-20 quarter and half notes Musical melodies consisting of 16 quarter notes 8 taxi drivers 8 controls Same as in the previous exp 12 lists of 15 street names 12 lists of 15 street names 9 players SELO 1780-2320 6 players SELO 2200-2370 8 players SELO 2100-2400 6 players SELO 2180-2390 4 players SELO 2200-2280 4 players SELO 1610-1710 40 chess positions and 20 transposed positions 42 chess positions and 20 transposed positions 6 chess positions and 6 transformed positions 8 chess positions 12 chess positions 6 masters SELO 2200-2400 6 medium-level SELO 1620-1850 6 players SELO 2050-2250 8 players SELO 1600-2100 3 masters SELO 2200-2350 3 medium-level SELO 1600-1850 6 chess games, 50 plies each 6 chess games, 50 plies each 6 chess games, 50 plies each 6 chess games, 50 plies each Participants Stimulus material

7.2. Stimuli and procedure

In every experiment, the stimuli were task-specific, such as chess positions or games, lists of street names, or musical patterns (Table 1). The detailed description of the material is found in the context of the original experiments. The meaningfulness of the material was varied by changing the organisation of the individual items, such as chess pieces in a position, street names in a list, or notes in a pattern. Using terms such as meaningful, random, and familiar is problematic in the context of expert memory: it

46

is difficult to define what constitutes meaning in the domain of expertise (Vicente & Wang, 1998). As long as it is unclear what makes a stimulus in a given domain meaningful, it is difficult to predict, a priori, which manipulations will affect expert performance. If the level of meaningfulness is defined post hoc from the performance level, it is a circular inference. In the present study, 'meaningfulness' refers to the structure of the stimulus. In chess, the pieces of the positions in Study II were a priori defined as functionally significant or functionally insignificant on the basis of chess book variations. The meaning of these positions was transformed by moving one functionally significant piece to another square (Figure 2) or by randomising the positioning of all the functionally insignificant pieces while the functionally significant pieces remained intact (Figure 3). Thus, the correct first moves and the main continuations were precisely the same for the original and the transformed chess positions. In taxi driving, the structure of the stimuli in Study III was manipulated by organising the streets in a list so that they formed drivable routes or impossible routes where the streets were randomly distributed in the list or structured in a specific way (Figure 4). In Study IV on music, the meaningfulness of the stimuli was the degree to which a pattern conformed to typical rules of tonality. Tonal melodies were inverted by randomising the order of notes (Figure 5) or by presenting the notes from the end to the beginning (Figure 6). These transformations disturbed the underlying harmony and rhythm. The procedure was an application of the protocol from blindfold chess where the moves are presented aurally using standard chess notation, instead of allowing participants to see the chessboard. In Studies I and II with chess, the positions and games were presented piece by piece, either visually or aurally. Thus, the participants were presented only the current move or one chess piece at a time and the complete view of the position had to be constructed in the mind. This presentation method was also applied to taxi driving and music, in Studies III and IV, respectively. In the domain of taxi driving, entire driving routes were not presented, but participants had to construct driving routes out of street names presented aurally one by one. In music, the melodies were presented visually, one note at a time. Therefore, the visual note pattern had to be constructed in the mind from seeing the sequence of individual notes. Furthermore, auditory information of the visual notes was not presented. The serial presentation of individual items is the essence of this method. It differs, however, from standard STM experiments in the greater number of presented items and in the domain specificity of the material. Several different dependent variables were used in the experiments. In chess, the immediate serial recall of moves and free recall and recognition of chess positions were both used. In chess problem solving, a master chess

47

player rated the generated solutions and counted the number of generated plies (half-moves) and episodes produced in the think-aloud protocols. In taxi driving, participants were required to recall the lists of street names in the correct serial order, but credit was given for correctly recalling street names in any order. In music, the notes needed to be recalled in the correct serial order, and the chunk scoring method (for details, see Clifton, 1986, pp. 9-12) described below was used. The presentation order of the stimuli and the order of experimental conditions were counterbalanced across the subjects.

7.2.1. Chess games and positions Study I. The stimulus material was 24 chess games, six in each experiment. They were randomly picked from chess magazines and were not familiar to the participants. The first 50 plies of the games were used in the experiments. In chess, a ply is a half-move, i.e., a move by either white or black. The term move refers to a pair of plies, i.e., a move of white and black. When referring to chess moves in a general sense (e.g., whites last move), the term move is used even though a half-move is referred to. We re-created a normal situation by asking participants to follow chess games presented on a 13.3 cm x 13.3 cm chessboard displayed on a PC screen. The plies were presented by showing the piece to be moved for 1 second in its original square on the chessboard. Thereafter, the piece disappeared and then reappeared 1 second later for 2 seconds in the destination square. Thus, only one piece was visible at a time. There was a 1 second interval between two successive plies, which resulted in a rate of one ply per five seconds. Three simultaneous games were presented from the beginning, alternately in 10-ply series. Participants were asked to recall the last position reached in the game after 30 plies and after 50 plies, successively for each of the three games, by placing normal chess pieces on a real chessboard. Finally, participants were asked to reproduce entire games by playing them, one by one, on a normal chessboard from the beginning up to the point where the participants could no longer remember the game. In Study I, the experimental conditions varied. In Experiment 1, half of the games were presented by using black dot symbols instead of standard chess symbols. In Experiment 2, half of the games were presented on a transposed board, which was divided into two halves along the midline and the halves were transposed so that columns e-h on a transposed board were situated where the a-d columns would be on a normal chess board, and vice versa (Figure 1). In Experiment 3, plies were presented using short algebraic notation where only the destination square of the piece is given (e.g., Ng3,

48

Ra4, that is, Knight to g3 and Rook to a4, respectively). The plies were presented visually on a computer screen or aurally from a tape at one ply every 4 seconds. In Experiment 4, the games were presented on a chessboard with standard chess symbols, as described above. In half of the games, participants determined the length of the pause between plies and games by clicking a mouse to see the next ply.

a)

b)

!""""""""# tMvWlVmT% OoOoOoOo% + + + +% + + + + % + + + +% + + + + % pPpPpPpP% RnBqKbNr% /)

!""""""""# lVmTtMvW% OoOoOoOo% + + + +% + + + + % + + + +% + + + + % pPpPpPpP% KbNrRnBq% /)

Figure 1. Normal (a) and transposed (b) chess positions used in Study I, experiment 2. In the transposed position, normal Ng1-f3 would be signed by Nc1b3. The games were presented on a transposed chessboard, but recall was performed on an ordinary chessboard. (Adapted from Saariluoma, P., & Kalakoski, V. Skilled imagery and long-term working memory. American Journal of Psychology, 110, 177-201. 1997)

Study II. The stimulus material was 96 middle-of-game and end-of-game positions selected from chess books. Depending on the experimental design, different transformations of the positions were constructed (Figure 2). In the irrelevant transformation, location of one functionally insignificant piece was moved to another square. In the relevant transformation, one functionally significant piece was changed. Functional significance was defined a priori on the basis of chess book variations. These transformations were used in experiments 1 and 2. In experiment 3, functionally insignificant chess pieces in other words, the functionally insignificant background was randomised and all functionally significant pieces in other words the functional figure remained intact (Figure 3). Thus, the correct first moves and the main continuations were precisely the same for original and transformed positions. The positions were presented piece by piece on a 13.3 cm x 13.3 cm chessboard on a PC monitor. One single piece appeared at a time for two seconds on an empty board and the interval before the appearance of the

49

next piece was 1 second. Thus, the rate was one piece per three seconds. All the white pawns were presented first and then all other white pieces were presented from top to bottom and from left to right. Thereafter, the black pieces were presented similarly.

a)

b)

!""""""""# +t+ Vl+% Ov+ + O % Om+ + +% + Mn+ W % + + + +% P +q+ + % bP + PpP% + + +rK % /)

c)

!""""""""# +t+ Vl+% +v+ + O % oOm+ + +% + Mn+ W % + + + +% P +q+ + % bP + PpP% + + +rK % /)

Figure 2. An example of transformations made for the positions in Study II, experiment 1. An original position (a); an irrelevant transformation of black pawn from a7 to a6 (b); and a relevant transformation of black pawn from g7 to f7 (c). (Adapted from Saariluoma, P., & Kalakoski, V.: Apperception and imagery in blindfold chess. Memory, 6, 67-90. 1998.)

!""""""""# +t+ Vl+% Ov+ +o+ % Om+ + +% + Mn+ W % + + + +% P +q+ + % bP + PpP% + + +rK % /)

In experiments 1 and 2, 30 positions were presented using this method, and participants were asked to encode them for later recognition. In experiment 2, the experimental group were asked to search for whites best move. The control group were asked to count the number of white pawns on white squares and black pawns on black squares, on the one hand, and the number of other white pawns on black squares and black pieces on white squares, on the other hand. Both groups were asked to give their answer during a 10-second interval between the positions. Thereafter followed the recognition task in which participants were shown complete positions. They were asked to say whether a position had been presented in the first part of

50

the experiment, and to give a reason for saying that it was not among the previously presented. In experiments 3-5, the positions were presented with the blindfold method described above (Study I). The task was to decide after each position the best move for white by reporting aloud all the thoughts that came into mind. After the problem solving task, participants were asked to reconstruct the original position on a chessboard with pieces.

a)

b)

!""""""""# t+ +t+l+% OvOm+ V % O + Wq+% + +o+ N % + P + +% + N P + % pP + Pp+% R + K +r% /)

!""""""""# + M +l+% +vKo+ V % + + +q+% +p+p+ N % w+ + P +% R O Pt+ % + +t+o+% P No+ Pr% /)

Figure 3. An example of transformations made for the positions in Study II,

experiment 1. In (a) and (b) the foreground of positions (marked pieces), correct first ply (Rh1h8), and main continuation (Rh1h8, Kxh8, Qh7++) are the same. In (a) the background pieces are located in an ordinary way, but in (b) the locations of the same background pieces are randomised. (Adapted from Saariluoma, P., & Kalakoski, V.: Apperception and imagery in blindfold chess. Memory, 6, 67-90. 1998)

In experiment 4, half of the positions were presented in a secondary task condition: participants determined every three seconds whether the corners of a letter they had been asked to imagine turned left or right. In experiment 5, fully visible presentation of the entire position and auditory presentations were used besides the blindfolded visual presentation. Several dependent variables were studied: recall of game positions and error percentages after 30 and 50 plies. In Study II, the main variables were the percentage of correctly recognised positions, the recall of positions after problem solving, and the number of generated plies and episodes in the problem solving tasks.

51

5 1 2 3 4 6

1 2 5 3 4 6 7

Figure 4. A schematic map and the organisation of the streets (1-7) on a map in
the two degrees of spatial continuity: a) route order, where the streets are spatially continuous, and b) map order, where the streets are not spatially continuous. Each arrow refers to a street on a list. Only the streets of two short lists are presented. In the randomised conditions, the street names were not presented in a successive order (1234567), but their order was pseudo-randomised (e.g., 6472135).

7.2.2. Lists of street names In Study III, twelve plus twelve series of 15 street names of Helsinki city were used as stimuli in two experiments. Six different types of lists were created: (1) streets in an order that form a spatially continuous route through part of the city of Helsinki; (2) these streets organised in pseudorandom order; (3) streets in an order that form a spatially structured path on a map but not forming a spatially continuous route; (4) these streets organised in pseudo-random order; (5) streets names belonging to the same semantic category, that is, related to the sea, to plants or to animals; and (6) streets arranged in alphabetical order (Figure 4). An audio tape presented street names on a particular list at the rate of one street per five seconds. After presentation of each list, the participants

52

were asked to recall as many items as possible, in correct serial order. The number of correctly recalled street names was scored.

7.2.3. Musical melodies In the pilot experiments for Study IV, the stimuli were musically wellformed tonal melodies picked from music theory books and their counterparts that were randomised by the experimenter (Figure 5). Another set of stimuli were childrens songs picked from songbooks and their mirror transformed counterparts, where backwards mirroring was used, and thus for example a note sequence d-a-f-g would transform into g-f-a-d (Figure 6).

a)

Figure 5. An example of a
complete musically well-formed note pattern (a) and a randomised version of a musical note pattern (b). A black dot refers to a quarter note and a white dot refers to a half note. (Adapted from Kalakoski, V.: Effect of skill level on recall of visually presented patterns of musical notes. Scandinavian Journal of Psychology, in press).

b)

a)

Figure 6. An example of a
complete musically well-formed note pattern (a) and a mirror version of a musical note pattern (b). A black dot refers to a quarter note and a white dot refers to a half note. (Adapted from Kalakoski, V.: Effect of skill level on recall of visually presented patterns of musical notes. Scandinavian Journal of Psychology, in press).

b)

In the experiments of Study IV, the melodies were transposed to a suitable key (e.g., C major or e minor) in order to avoid sharp and flat signs in the visual array to be recalled. In Experiment 1, the stimuli consisted of 18 simple tonal melodies picked from foreign musical songbooks for elementary-level music education, and their 18 mirror counterparts. The

53

melodies were simplified to consist of 10-20 quarter and half notes. These melodies and their mirror counterparts significantly differed from each other in subjectively rated musical goodness, where non-musicians estimated on a 5-point likert scale how good they found the auditorily presented melodies. In Experiment 2, the 24 note patterns were based on childrens songs and traditional tonal melodies picked from a songbook used in elementary and secondary school. Only 16 filled dots, representing quarter notes, were used. Half of the presented melodies were mirror counterparts, but the effect of stimulus type (melody vs. mirror counterpart) was not studied in this experiment. The method of subjectively estimating stimulus goodness was used in the selection of stimuli (see also Gathercole, 1995; Gathercole, Willis, Emslie, & Baddeley, 1991; Povel & Jansen, 2001). It was assumed that as in language, the rated goodness of stimuli in the domain of music is also related to prelearned musical knowledge and skills (Gathercole, 1995). In every experiment, a single note appeared at a time on a short stave in the centre of a 21 PC screen. Only five lines were presented, with filled and open dots for quarter and half notes, respectively. No sharps or flats were involved. Every dot was visible for 2 seconds in the same horizontal location on the stave and each note was followed by a pause of one second. After presentation, the participants were asked to recall the notes from the beginning in the correct serial order by drawing all the dots they could remember on an empty stave on a recall sheet. In experiment 2, tonal patterns were also presented using written note names at the same rate as in the note presentation. Note names consisted of the letters c, d, e, f, g, a, h, which are the symbols used for note names in the Finnish language, and the digits 1 or 2, indicating the octave (e.g., c1, e1, g2). The participants were instructed to try to hear the melodies in their minds while storing them. Recall of the note patterns was scored using a chunk scoring system. This system has been devised to be sensitive to partially correct recall in note reading tasks, and to be highly correlated (r > .90) with scoring notes in the correct serial position (Clifton, 1986). One point was given for correct recall of three successive notes, and then one point for each additional correct note, for instance, three correct notes in succession would result in one point, four successively correct notes would result in two points, and three successively correct notes together with three successively correct notes in another section of the series would provide 1+1 (=2) points.

54

8. Summary of the results

The main variables and results of the experiments are presented in Table 2. The effects reported below are based on the statistical significances found in the context of the original experiments. The figures 7, 9, and 10 present the effect sizes d (Cohen, 1988), which are comparable values across the experiments. The effect sizes were calculated from the values (F, t, M, SD, SEM) available in the original publications and the data. These numbers show the relative sizes of the effects over the experiments in the present study. For example, an effect size of 0.5 for stimulus surface features effect would indicate that the distribution of scores for the one type of stimulus surface features overlaps 67 % with the distribution of scores for another type of stimulus surface features. An effect size of 1.0 for skill effect would indicate that the distribution of scores for the expert group overlaps 45 % with the distribution of scores for the non-expert group. Similarly, the effect sizes of 2.0, 3.0, 4.0, and 7.0 indicate an overlap of 19 %, 7%, 2 %, and 0.02 %, respectively.

8.1. The effect of skill level

The results showed apparent skill effects in all of the studied domains. This result suggests that domain-specific knowledge accompanies mental imagery. The domain specificity of the mechanisms underlying experts images was also indicated by the interaction between skill-level and the meaningfulness of the material: only experts benefited if the material was meaningful. (Figure 7) In chess, highly skilled masters outperformed medium-level players in recalling game positions after 30 and 50 plies of the games that had been presented to them (Study I, Experiments 1 and 2). They also had smaller error percentages than did medium-level players. Moreover, masters outperformed medium-level players in recalling chess positions after a problem-solving task, and they also had smaller error percentages. The skill effect was obvious in every presentation condition, whether stimuli were presented in a fully visible manner, or with the visual blindfolded method or aurally (Study II, Experiment 5). Skill level also affected problem solving. Masters had a higher percentage of correct first moves, and a greater number of generated plies and episodes than medium-level players did. An interaction was also found between skill level and speed of encoding (Study I, Experiment 4). Masters recall of game positions after 30 and 50 plies was higher than medium-level players only in the limited encoding time condition. In contrast, in the self-paced condition, the performance of

55

two skill groups was comparable. Thus, medium-level chess players were almost able to reach master-level performance if they were allowed to selfpace the presentation of the next stimulus. However, if the time used per ply was controlled, masters recalled significantly more pieces of the positions after 30 and 50 plies than medium-level players. In the domain of taxi driving (Study III), the overall effect of skill level was evident in both experiments. Taxi drivers recalled more street names than the control participants. Moreover, the interaction between skill level and degree of randomness of the street names in the lists showed that experts outperformed novices only when the taxi drivers were able to use domain-specific and ecologically relevant information in the street memory task; that is, when the lists consisted of streets which were spatially close to each other or formed a spatially structured path on the city map. There was no difference in the memory performance of taxi drivers and the control group when the streets were located far apart and were presented in random order, or lists consisted of street names which were organised according to other rules than spatial, e.g. by category or in alphabetical order. These results support the basic finding of domain-specificity of expert performance in the new domain of taxi driving. In music (Study IV), experts outperformed control participants in every experiment as well as in the preliminary experiments. They recalled more items than the control participants from visually presented patterns of musical notes (preliminary experiments and Experiment 1), and from written note names (Experiment 2). A significant interaction between skill level and stimulus type was found in the second preliminary experiment and in Experiment 1. It indicated different effects of stimulus type for the two skill groups. However, the skill effect was found for all types of stimuli. Results concerning the distribution of recall errors (Figure 8) in the letter-name condition indicated a qualitative difference between the two skill groups (Study IV, Experiment 2). Musicians made more errors where the notes were displaced 1-3 steps above or below the original notes, and non-musicians made fewer such errors than expected if the errors had been distributed similarly in the two skill groups. In contrast, musicians made fewer errors in the digits indicating the octave, and non-musicians made more such errors than expected. In sum, the effect sizes for the main effect of the skill level on recall (Figure 7) were 2.4-3.6 in chess, 3.0-3.8 in taxi driving, and 1.3-2.8 in music. Skill level also had a significant interaction with the structure of the stimuli in taxi driving and music where it was studied. The results also showed that the size of the skill effect in taxi driving depended on the stimulus type.

56

Table 2. Variables studied and summary of significant effects Exp. Study I 1 Skill level % correct Presentation: Error % Blind-fold vs. Dot Recall session Presentation: Normal chess vs. Transposed Recall session Presentation: Auditory vs. Visual Skill level Presentation: Regular vs. Selfpaced % correct Error % Masters > Medium: % correct, Error % Small non-significant effect of presentation: Chess > Dot First recall > Second recall Normal board > Transposed board First recall > Second recall Independent Dependent variables variables Significant interactions and main effects

% correct Error %

No effect of presentation First recall > Second recall

% correct Error %

Session * Presentation Skill * Session * Presentation Masters > Medium, in the regular condition First recall > Second recall, in the control condition for medium-level players Self-paced > Control, in the medium players

Study II 1 Type of position change: Same, Irrelevant, Relevant, Different Type of position change: Same vs. Irrelevant vs. Relevant vs. Different Orienting task: Problem solving vs. Counting Type of position: Normal background vs. Randomised background % of correct answers Reaction time % of correct arguments Correct answers: Relevant change > Irrelevant change Correct arguments: Relevant change > Irrelevant change

% of correct answers Type of position * Orienting task % of correct Problem solving group: Correct answers: arguments Relevant change > Irrelevant change Counting group: no effect

% of correctly recalled pieces Error % % of correct first moves Reaction time

Normal background > Randomised: Correct recall, Error %, Correct first moves Normal background < Randomised: Reaction time

57

Table 2 cont. Exp. 4 Independent variables Presentation: Control vs. Interference Dependent variables Generated plies Unique first plies Generated episodes Number of pieces mentioned Recall % of mentioned % of correct first ply Correct recall Error % Correct recall Error % Correct first moves Generated plies Unique first plies Generated episodes Significant interactions and main effects Control > Interference: Generated plies, Unique first plies, Generated episodes, Number of pieces No effect: % of correct first ply, Recall % of mentioned, Correct recall, Error %

Skill level Presentation: Fully visible vs. Visual blindfold vs. Auditory blindfold

Skill * Presentation Masters > Medium: In every variable except unique first plies Presentation: Blindfolded visual = auditory, Fully visible > Blindfolded: Generated plies, Generated episodes Unique first plies: Medium: Fully visible > Blindfolded, Masters: no effect

Study III 1 Type of list: Recall level Route order vs. Route random vs. Map order vs. Map random Type of list: Recall level Route order vs. Route random vs. Semantic vs. Alphabetical Skill level * Degree of randomness Skill effect: Route order, Route random, Map order No skill effect: Map random Taxi drivers: Route order > Route random = Map order > Map random Control: No effect Skill level * Degree of randomness Skill effect: Route order, Route random No effect: Semantic, Alphabetical Taxi drivers: Route order > Route random > Semantic = Alphabetical Control: Route order > Alphabetical

Study IV Prel. Skill level Recall level exp. Type of melody: Musical vs. Random vs. Mirror 1 Skill level Recall level Type of melody: Musical vs. Mirror Skill level Presentation: Note vs. Letter Recall level Distribution of errors Musicians > Non-musicians Musical > Random Skill level * Stimulus type (prel. exp. 2): Only musicians: Musical > Mirror Skill level * Stimulus type Musicians > Non-musicians Musicians: Musical > Mirror Non-musicians: Musical = Mirror Recall: Musicians > Non-musicians Note presentation = Letter presentation Distribution of errors: Qualitative difference between the skill groups

58

CHESS I; 1 I; 4 I; 4: speed of encoding I; 4: skill*speed II; 5 recall II; 5 correct first II; 5 N of plies

TAXI DRIVING III; 1 III; 1: skill*structure III; 1 route III; 1 route random III; 1 map III; 1 map random III; 2 III; 2: skill*structure III; 2 route III; 2 route random III; 2 semant. III; 2 alphab.

MUSIC IV; pre1 IV; pre1: skill*structure IV; pre2 IV; pre2: skill*structure IV; 1 IV; 1: skill*structure IV; 2 0 1 2 3 4 5 6 7 8

Effect Size (Cohen's d)

Figure 7. The effect sizes for the effects of Skill level and Skill level * Stimulus structure interactions on performance over the experiments and the effect size for the speed of encoding (Study; Experiment: Condition). For example, I; 4 refers to the main effect of skill level in experiment 4 of Study I, and III; 2: route refers to the skill effect for route organised lists in experiment 2 of Study III.

59

Visual notes: musicians

1-3 steps position

Letters: musicians
1-3 steps position

order

order octave

miscellaneous

miscellaneous

Visual notes: non-musicians

1-3 steps position order

Letters: non-musicians
position 1-3 steps order octave

miscellaneous

miscellaneous

Figure 8. Percentage of errors for musicians and control participants in the visual note and the letter name conditions. The error categories were a) notes displaced 13 steps above or below the original notes (1-3 steps), b) notes displaced in the leftright orientation in different serial positions (position), c) notes in incorrect successive order (order), d) notes with a correct name but a wrong octave (octave), and e) notes that could not be classified according to the previous rules (miscellaneous).

8.2. Variables affecting the construction of skilled images

To understand the mechanisms underlying skilled imagery, the effects of several variables were studied. The effect of surface features reflects early encoding processes and perceptual chunking, and these were varied in the domains of chess and music (Figure 9). The structure of the stimuli is related to the meaningfulness and the possibility to use pre-learned LTM knowledge in conceptual chunking, and it was studied in all three domains (Figure 10).

60

8.2.1. The surface features of the material In chess, the mechanisms related to early information encoding in representation construction were investigated in Studies I and II. The presentation modality did not affect recall of the positions after 30 and 50 plies, as the percentages of correctly recalled pieces and errors were comparable in the visual algebraic notation and the auditory presentation conditions (Study I, Experiment 3). Neither did presentation modality affect the percentage of correctly recalled pieces after the problem-solving task: The number of correctly recalled pieces and the error percentages were comparable in the fully visible, blindfolded visual, and blindfolded auditory conditions (Study II, Experiment 5).

CHESS I; 1 I; 1: Skill * Surface I; 2 I; 3 II; 5 recall II; 5 correct first II; 5 N of plies Effect of secondary task II; 4: N of plies II; 4: recall

MUSIC IV; 2 IV; 2: Skill * Surface 0 1 2 3 4 5 6 7 8

Effect size (Cohen's d)

Figure 9. The effect sizes for the effects of stimulus surface features, skill level * surface feature interaction over the experiments, and the effect of secondary task on performance in chess (Study; Experiment: Condition). For example, II; 5: recall, refers to the effect of stimulus surfaces features on recall of chess positions in experiment 5 of study II, whereas II; 5: N of plies refers to the effect of stimulus surfaces features on the number of generated plies in that experiment.

61

Furthermore, surface features related to the colour and kind of the presented pieces did not affect the recall. The dot presentation condition impaired recall levels a little, but non-significantly, and only for the medium-level group in comparison to the visual blindfolded presentation in which colour and kind of pieces were shown (Study I, Experiment 1). Experiment 4 of Study II further showed that the visuo-spatial secondary task conducted during the problem-solving phase that followed the presentation of chess pieces did not affect recall level or error percentage. In the present study, the only significant effect of presentation method on recall for chess was for the transformation of the spatial locations of the pieces. The percentages of correctly recalled pieces after 30 and 50 plies were lower and error percentages were higher when the board was divided along the midline (and the halves transposed) compared to an intact board (Study I, Experiment 2). The effects of representation construction were also studied on the variables indicating the process of problem solving. In these variables, the effect of the presentation format was found to be significant (Study II, Experiment 5). Chess players generated more plies and episodes in the fully visible condition than in the visual and auditory blindfolded conditions. Moreover, medium-level players generated more unique first plies in the fully visible condition than in the two blindfolded conditions. The performance in the two blindfolded conditions was comparable. Moreover, the visuo-spatial secondary task during problem solving decreased the number of generated plies and episodes and the number of unique first plies (Study II, Experiment 4). In music, the role of surface features was studied in Experiment 2. The results showed no effect of the presentation mode. Musicians and the control group recalled visually presented notes as well as the letters referring to note names. To sum up, the sizes of the stimulus surface feature effects were mainly below 1. The exceptions were the effect of divided board (Study I, Experiment 2) and the effect of presentation format on the number of generated plies and episodes in chess (Study II, Experiment 5). The size of these effects was below 2.

8.2.2. The structure of the stimuli In chess, Study II concentrated on the effect of the structure of the stimuli. In Experiments 1 and 2, the type of transformation that was made of the original chess positions affected the later recognition of positions. The minimal transformation of one single piece in a functional figure or background of the position was critical. The results showed that the percentage of correctly recognised significant changed positions was higher

62

than that of insignificant changed positions. Chess players were also able to give more correct arguments for the difference between original and transformed positions in the case of functionally significant than functionally insignificant changes. Furthermore, in Experiment 2, a statistically significant interaction between the type of transformed position and orienting task was found. When the task was to search for whites best move during the learning phase, the percentage of correctly recognised transformed positions was higher for the significant change than for the insignificant change positions. In contrast, if the orienting task during the learning phase had been counting the number of pieces, no difference in the correct recognition of the two types of minimally transformed positions was found. In Experiment 3, the task was to decide the best move for white, and thereafter to reconstruct the position. There were two types of positions, with background pieces located as they would appear in a normal game or in a randomised and thus unfamiliar and impossible positioning. Chess players recalled more pieces and had fewer errors in the ordinary positions than in the background-randomised positions. Furthermore, when the background was randomised, they spent more time problem solving, but still had fewer correct answers. In the taxi driving experiments (Study III), the degree of spatial randomness affected recall level in the expert group. Taxi drivers recalled more items from lists that formed a spatially continuous route (route order lists) than if these street names were organised in random order (route random lists), or if the streets were spatially organised on a map but did not form a continuous route (map lists). Moreover, they recalled fewer items if the map ordering of streets was organised in a random order (map random lists) in comparison to the other list types. In the second experiment, the recall levels of lists with semantic organisation and alphabetical organisation were lower than in the route order and route random lists and comparable to the map random lists in the first experiment. In the control group, recall of route and map lists were comparable, and the fewest items were recalled from alphabetically organised lists.

CHESS II; 1 II; 2: counting II; 2: probl. solv. II; 3 TAXI DRIVING III; 1 III; 1: taxidrivers r vs. rr III; 1: taxidrivers r vs. m III; 1: taxidrivers r vs. mr III; 1: control r vs. rr III; 1: control r vs. m III; 1: control r vs. mr III; 2 III; 2: taxidrivers r vs. rr III; 2: taxidrivers r vs. s III; 2: taxidrivers r vs. a III; 2: control r vs. rr III; 2: control r vs. s III; 2: control r vs. a MUSIC IV; pre1 IV; pre2 IV; 1 IV, 1 musicians IV, 1 control 0 1 2 3 4 5 6 7 8

Effect size (Cohen's d)

Figure 10. The effect of stimulus structure on performance over the experiments (Study; Experiment: Condition). For example, III; 1, refers to the effect of stimulus structure on recall of street names in the experiment 1 of the study III, whereas III; 1: taxi drivers r vs. rr, refers to the effect of stimulus structure in the taxi drivers group when r (route) and rr (route random) lists are compared (m = map, mr = map random, s = semantic, a = alphabetical).

64

In music (Study IV), the effect of the meaningfulness of the material for later recall was studied in preliminary experiments and Experiment 1. The results showed that only musicians were able to recall the musical melodies better than their mirror counterparts, regardless of whether the stimuli were familiar melodies (second preliminary experiment) or novel tunes (Experiment 1). In the first preliminary experiment, musical patterns found in actual melodies were recalled by all participants better than randomised patterns of visual notes. Results concerning the subjective estimations of stimulus material showed that randomised melodies differed from normal melodies not only in auditory goodness but also in visual goodness, whereas the melodies and their mirror counter parts only differed in auditory goodness. In sum, the sizes of the stimulus structure effects were mainly below 1 for non-experts. The variation in effect sizes in expert groups was large, the range being mainly from 1.5 to 7.

65

9. Discussion
The central question for the present thesis was whether facilitating longterm memory knowledge and skills in working memory imagery tasks are primarily based on perceptual chunking or whether they rely on higher-level conceptual knowledge. A new method was developed to investigate this issue. The results showed large skill effects in all of the studied domains; chess, taxi driving and music. Furthermore, skill level interacted with the meaningfulness of the material. The effects of stimulus surface features were small, except for the effect of divided board in chess and for the effects of stimulus surface features on problem solving. In contrast, the sizes of stimulus structure effects were large. These results will be discussed in more detail next in terms of the hypotheses.

9.1. Skill level affects the incremental construction of mental images

A significant effect of skill level was found in every studied domain. In terms of the hypothesis, the better performance of the experts indicates that prelearned knowledge plays a crucial role in the construction of mental imagery. Moreover, the skill effect and the performance levels in the three domains of the present studies were comparable with those reported in other studies using different methods. In Studies I and II the chess players obtained comparable recall levels as in the previous studies, in control conditions 60-90% correct pieces of the positions depending on the skill level. This result replicates the skill effect found in the earlier study on blindfold chess (Saariluoma, 1991) and in other studies presenting chess stimuli in unrelated groups (Cooke, Atlas, Lane, & Berger, 1993; Frey & Adesman, 1976; Saariluoma, 1989). In Study IV on music, the recall levels indicated correct recall of 3-4 successive notes or two groups of three notes in the control group, and 4-8 successive notes or several groups of three or more successive notes in the expert group. The results of previous studies on musical memory using different presentation methods showed mean recall levels of about 1-2.5 notes for non-musicians, and about 2-6 notes for musically experienced participants (Halpern & Bower, 1982; Meinz & Salthouse, 1998; Roberts, 1986; Sloboda, 1976). To my knowledge, comparable studies have not been conducted before in the domain of taxi driving. In Study III, mean recall level of taxi drivers was about 13 street names in the best recall condition. The control group was able to recall about 5-6 items from the lists.

66

The results for chess, Studies I and II, are consistent with the earlier studies suggesting that chess players use visual imagery in chess memory and problem solving (Robbins et al., 1996; Saariluoma, 1991, 1992c). Studies III and IV introduced two new domains of expert imagery. Our results involving taxi drivers memory and their introspective comments suggest that they use pre-learned knowledge concerning the large-scale visuo-spatial environment to simulate driving in the city. The taxi drivers were therefore able to outperform the control group in the immediate recall of verbal lists of street names. Presumably, the expert musicians in Study IV were also able to construct an internal auditory representation from the visually presented notes, as was requested in the instructions. They were able to transform visual notes into auditory images, which enabled them to outperform the control group at recalling successively presented visual notes. It is difficult to explain the results without assuming auditory imagery, and the musicians subjective experience also suggests that they used auditory imagery. Other studies provide additional evidence that auditory imagery can be evoked with visual presentation of a complex note pattern (Brodsky, Henik, Rubinstein, & Zorman, 2003). It is, therefore, reasonable to assume that auditory images were used in the present task, too. The corresponding presentation methods used in the 5 studies of this thesis made it possible to compare the resulting patterns in three different areas of expertise. However, this comparability is not complete and has several limitations. For example, although the effects sizes for skill level effect (see Figure 7) were great and coherent across the domains, they also varied from study to study. Therefore it is not possible to infer whether these variations result from different mechanisms underlying different domains or from the non-comparability of the degrees of expertise between the domains. This issue is related to a more general problem of researching expertise: there is no objective cross-domain measure of the degree of expertise. Therefore, it was not possible to ensure that the expert chess players, taxi drivers, and musicians studied had similar levels of expertise, and the sizes of skill effects in the three domains are not strictly comparable. Nevertheless, they reflect the general principle that skill level has a great impact, especially when the material is meaningfully organised. Furthermore, the skill level of non-experts varied substantially in the three domains. Whereas novice chess players had much experience and an official rating of their chess skill, the non-musicians were novices who were not able to sight-read notes at all. Thus, the difference in the performance of the two music skill groups could mainly reflect two different ways of performing the task (for example, auditory imagery in the musician group, and visual and verbal rehearsal in the non-musician group). Similarly, in the taxi driving domain, different background knowledge of the city influences the rehearsal strategies; taxi drivers can simulate driving the routes, whereas

67

the control participants were mainly able to use verbal rehearsal strategies. Thus, only in chess were the two groups of participants in principle able to rely on similar strategies whereas in the other two domains the novices had no access to expert ways of processing the stimuli. Although the comparability between the studies is not complete, the results demonstrate the significance of the level of expertise in constructing representations in all three domains. It should also be noted that different dependent variables, those related to recall, recognition and problem solving, may be differently sensitive to skill effects. In future studies, a detailed analysis of the nature of these variables would increase our understanding of the mechanisms underlying skill effects. If the main objective of such studies is a detailed comparison of performance between several domains, the issue of objective cross-domain measures of the degree of expertise should be handled more carefully than in the present studies. In sum, the method used with the present research produced recall levels and skill effects that are comparable to those of previous studies in chess and music cognition. Consequently, the present method appears to be reliable for studying general principles underlying expert performance in mental imagery construction tasks. Although the evidence for mental imagery in the three domains is only indirect and based on subjective reports of the participants, the sequential presentation of stimuli and the patterns of the results presuppose mental imagery. The results demonstrate the genuine excellence of experts, even in a novel domain-relevant task and, thus, demonstrate their ability to adapt to new task demands.

9.1.1. Experts attune to task relevant constraints The interaction between skill level and the meaningfulness of stimuli studied in taxi driving and music further demonstrated the domain specificity of skill effect. The results showed that taxi drivers outperformed novices with lists of street names that formed spatially continuous routes and lists forming a spatially structured path on a map, and even with randomly presented street names if they were randomised from continuous routes. In contrast, recall levels were comparable in the taxi drivers group and the control group when the lists were randomised from a spatially noncontinuous order or if the streets were organised by a semantic category or alphabetically. Our results show that the level of recall is a function of the number of goal-relevant constraints (for example, whether a route is drivable or not) that experts can take advantage of to structure the stimuli. Vicente and Wang (1998) have suggested an abstraction hierarchy for the domain of chess, as described in the introduction. Applying the CAH to taxi driving, a

68

possible abstraction hierarchy could be the following. In taxi driving, the obvious general purpose of the business is to take the customer from point A to point B. The level of the abstraction hierarchy dealing with strategies can define, for example, that if there are traffic jams, a longer route may be faster. At the level of tactics, it is possible, for example, to choose the shortest or the fastest route between two addresses. At the next level, there are rules of driving (e.g. one-way streets) that define drivable routes. At the lowest level, there are the streets and the spatial organisation of the city. It is evident that categorical and alphabetical organisation is not in line with the task constraints that taxi drivers face when they generate the best routes for travelling between locations in the city. Although semantic relations between items have been shown to be a good mnemonic for verbal material, taxi drivers in the present study got no extra memory advantage from semantically organised material compared to control subjects. In contrast, musicians outperformed the control group with every type of melody. An unexpected finding was the lack of interaction between skill level and stimulus type when melodies and their randomised counterparts were studied; results showed that musicians outperformed the control group even with randomised melodies. Halpern and Bower (1982) got similar results when using fully visible patterns of musical notes. They suggested that musicians are able to find meaning in any configuration of visual notes. This suggestion echoes theories on expert memory that also predict expertise effects also for randomised stimuli if there is enough time to construct meaningful configurations (Gobet & Simon, 1996a; 2000; Gobet & Waters, 2003; Saariluoma, 1989, 1995). Study IV, replicating the Halpern and Bower (1982) study, showed that not only musicians but also the control group performed better with musical melodies than with randomised sequences of pitches and rhythms. Halpern and Bower proposed that randomisation affects not only the musical meaning of the melodies but also their visual goodness. The present study showed direct evidence for this claim: the subjective ratings of the stimulus material indicated that randomisation affected not only the auditory, but also the visual goodness of the melodies. Although the nonmusicians were not able to use auditory imagery in the task, they presumably tried to form a visual image or used verbal rehearsal and were thus able to better recall melodies than their randomised counterparts. However, only the musicians performance was impaired if the notes in the melodies were presented in the reversed serial order from end to beginning. This changed the melodies auditory and musical but not visual goodness (because, when reversed, harmonic and rhythmic properties, for example, do not progress as is usual in tonal harmony). The results suggest that musicians and non-musicians were attuned to different stimulus features

69

and this affected their performance, as was shown in the effect of stimulus material. The results showed that experts in the studied domains were able to use domain-specific or ecologically relevant information in tasks that are not the normal daily activities of the experts (Ericsson, Patel, & Kintsch, 2000; Vicente & Wang, 1998). Therefore, the present results shed light on domains where exceptional memory performance is a by-product of existing skill. The results are in agreement with the ecological approach to cognitive skills - the approach based on the assumption that exceptional performance is grounded in adaptation to the constraints imposed by the environment (Ericsson & Lehmann, 1996; Vicente & Wang, 1998). Although the general principle of the attuning to task constraints proposed by the CAH is applicable to the present results, it does not describe the mechanisms underlying expert exceptional performance. Next, then, the mechanisms underlying expert imagery are discussed from the viewpoint of expert memory theories.

9.2. Perceptual chunking has a minor role in imagery construction

Since the memory and problem solving tasks used in the present research presumably required mental imagery, it was hypothesised that perceptual features may have an important role in representation construction, as suggested by mental imagery research and by expert memory research that emphasises perceptual pattern matching (Gobet & Simon, 1996a). On the other hand, since the stimuli were presented one-at-a-time, not as a whole pattern, the task itself did not necessarily facilitate direct pattern matching, but conceptual chunking. Theoretically, it is important that an expertise effect was found in the task of incremental construction of representations. This finding indicates that the ability to chunk incoming information into meaningful units does not require that complete familiar patterns are accessible for encoding. However, it is possible that the core mechanism underlying expert performance in the present task is still based on matching perceptual chunks in LTM with successively presented items. The role of perceptual chunking was studied by manipulating the surface features of the stimuli. Results showed that experts performance was not impaired, for example, when the chess pieces were replaced by black dots that moved according to the rules of the game. In music, recall levels in notated versus letter conditions were comparable, although in the letter presentation condition the visual cues of the musical stave were not available. Apparently, experts were able to rapidly transform a

70

representation format to another that allowed the use of pre-learned knowledge. This result also indicates experts ability to use pre-learned knowledge and skills even though the material is not presented using typical visual features. Other results of the present studies also indicate that the better recall of experts cannot be explained solely by perceptual visual chunking. In chess, performance in memory tasks with blindfolded visual encoding was found to be at about the same level as in the auditory blindfolded condition and in the whole board visual presentation. In taxi driving, experts outperformed novices not only when lists formed a drivable route, but also when streets were located near each other or formed a structured non-continuous path. Thus, taxi drivers superior performance could not be based on the direct recognition of drivable routes. In music, the effect of skill level was evident not only for familiar melodies but also for when unfamiliar and musically meaningful stimuli were used. This indicates that the process of utilising pre-learned knowledge was not based on the direct recognition of familiar melodies. Transforming the spatial locations of the two halves of the chess board impaired recall performance. The interpretation that transforming spatial locations hinders the mapping of LTM perceptual chunks on the mental image has been put forward in the context of earlier findings of the importance of encoding spatial location in chess (Gobet & Simon, 1996a; Saariluoma, 1984). The presupposition is that, for chess experts, the perceptual chunking process in the minds eye proceeds as it does with perceptual stimuli (Campitelli & Gobet, 2005). Another possible interpretation is that when spatially transformed chess boards are used, players have to mentally re-transform the board before the game becomes meaningful. This strategy requires visuo-spatial WM resources and will, therefore, lead to impaired performance. In sum, impaired performance with spatially transformed positions does not necessarily indicate perceptual chunking in the minds eye. Instead it might imply that spatially transformed positions require more visuo-spatial or executive WM resources than normal positions do. A noteworthy finding was that the chess experts did not encode all the pieces in the positions equally when they constructed representations: a functionally significant change of one chess piece was detected more easily than a change of a functionally insignificant piece. This suggests that experts representations consist of perfectly encoded functional figures but that the background pieces are not perfectly represented. This effect was stronger in the condition with the semantic orienting task and disappeared with non-semantic orientation. The effect of the orienting task replicated the earlier findings on the recall and recognition of chess positions (Goldin, 1978; Lane & Robertson, 1979). These results show that familiar perceptual

71

patterns are not automatically found; the player must attune to the relevant constraints (Vicente & Wang, 1998). Furthermore, the background pieces that did not have a role in the game continuation still affected the construction of the representation: randomised background pieces impaired the construction of representations and problem solving. These results indicate that also the irrelevant pieces have a role in representation construction. If performance was based only on direct pattern matching, the pieces that are not part of the significant patterns should not affect performance. The present studies suggest that experts adaptation to task demands is very fast, and that they are able to immediately rely on strategic control if perceptual chunks are not available. Moreover, when the time for encoding the games was unlimited, medium-level players were able to improve their recall to the level of the masters. The decrease in encoding and study time with practice is predicted by the theory of LTWM and the template theory. The results discussed so far suggest that, although perceptual chunking may play a critical role in experts cognition (Gobet, 1998), other accounts are needed to explain the present results on the incremental construction of mental images.

9.3. Conceptual chunking underlies skilled imagery

The present research offers evidence for a slower, problem solving-like chunking process that is used in tasks where fast perceptual chunking cannot be easily used (Chase & Simon, 1973). These results are in line with the knowledge-based frameworks that have shown that the depth of processing affects recall of chess positions (Goldin, 1978; Lane & Robertson, 1979), and that there are qualitative differences between experts and novices representations (Adelson, 1984; Chi, Glaser, & Rees, 1982). Chess players ability to detect functionally significant changes in the positions of pieces and the role of background pieces in constructing representations both support the role of high-level conceptual knowledge. Chess players construct representations by interpreting the position of pieces rather than by directly recognising which items belong to the figure and which constitute the background. The results also suggest some qualitative differences between skilled and less skilled players. Although skill level did not significantly interact with the presentation format, medium-level players performance in the dot condition seemed to be impaired. Gobet and Waters (2003, p. 1091) have assumed that appearance-related constraints are of equal value to expert and weaker players. In contrast, some earlier studies have shown that skill level also affects performance at the perceptual level. Masters are faster than

72

medium-level players in chess piece classification (Saariluoma, 1984) and chess expertise affects performance in the moving spot task (Attneave & Curlee, 1983) where dots are replaced by chess pieces (Bachmann & Oit, 1992). In the present study, although both masters and medium-level players were presented chess pieces that looked the same, the results suggest differences in what they saw when the dot presentation was used. Experts outperformed less skilled players, which implies that they were able to use higher-level abstraction hierarchies even though the lower-level features were not present. This indicates that either the lower-level features are not necessary in skilled imagery or that they are constructed from higher-level knowledge when not available perceptually. Furthermore, the results on the effect of skill level on the distribution of recall errors in music indicate that there was a qualitative difference between musicians and non-musicians in the way they represented the stimuli. In the letter-name condition, skill level affected the distribution of errors as hypothesised. Displacement of notes one step higher or lower on the stave is a natural musical and visual error but in the letter condition only musicians made more such errors. The results support the conclusion that only musicians were able to code the letters aurally as pitches and melodies, and that this strategy was the base of their superior memory. In contrast, the non-musicians confused octaves more often than the musicians did, because for them there was no semantic information involved in the digits referring to octaves. Thus, non-musicians seem to have encoded the material as visual figures or verbal descriptions rather than as auditory melodies.

9.3.1. Mental images are used as retrieval structures The results of the present studies show that mental imagery is an efficient mnemonic system for experts, and that knowledge and conceptual chunking underlie the construction of images: chess players were able to combine individual pieces in their mental imagery into coherent positions, taxi drivers connected the street names into spatial routes, and musicians transformed notation into auditory images and were thus able to find meaningful configurations. The literature on expert memory indicates that the retrieval structures proposed by the theory of LTWM are often spatial. Accordingly, our results on chess and taxi driving suggest that part of LTWM might be visuo-spatial. However, visuo-spatial encoding of individual items is not sufficient for expert memory; semantic or other associations are also required. For example, previous studies have shown that an experienced waiter not only encoded orders according to the spatial location of the customer; he also used the semantic category of the items in constructing memory units

73

(Ericsson, 1988; Ericsson & Polson, 1988). Similarly, in chess, recalling multiple chess positions is based on semantic associations between the positions (Gobet & Simon, 1996b). The present results suggest that although taxi drivers were using a spatial retrieval structure, the superior memory of taxi drivers is not based on associating individual items with the locations on the map: associating verbal street labels only with the locations on the map did not guarantee superior memory unless the items constituted a logical spatial path. Our studies show that the cognitive mechanism underlying the taxi drivers superior memory is based on a retrieval system that structured the individual items in relation to each other. However, in this case, only spatial relations between the items, not semantic or alphabetical mnemonics, were crucial. Thus, imagining only unrelated pieces of information is an insufficient mnemonic; only when individual pieces of information can be chunked can the capacity limits of WM be exceeded. This phenomenon has also been conceptualised as a process of relational coding of individual items, and this is considered one of the mechanisms underlying the effectiveness of mental imagery mnemonics (Bower, 1970; Marschark & Surian, 1989, 1992). Also, the present results suggest that in experts exceptional memory, an effective mental imagery mnemonic requires a retrieval structure that organises individual items in relation to each other.

9.4. Skilled images are constructed in working memory

The piecemeal presentation method used in the present studies creates a demanding WM task. The individual items had to be kept active in WM before access to knowledge in LTM could add encoding support. It is evident that only skilled people are able to construct complex mental images because pre-learned conceptual knowledge enables them to chunk information and to construct visual, spatial, and auditory images that require a capacity above the limits of WM (Saariluoma, 1991). The effect of skill level on immediate recall for all of the studied domains indicates the ability of experts to exceed the limitations of WM. The capacity limitations of an experts WM are a problem only for the encoding and manipulation of information. After a memory representation has been constructed, it is stored in the LTM or LTWM, where it cannot be interfered with (Charness, 1976; Ericsson & Kintsch, 1995; Robbins et al., 1996; Saariluoma, 1992c). The results of the present study further showed that although the visuo-spatial secondary task did not impair recall of chess positions, it impaired problem solving. Thus, although visuo-spatial WM is not crucial after the positions have been stored in LTM or LTWM, problem

74

solving in chess requires visuo-spatial WM resources (Robbins et al., 1996). There is also evidence that the central executive of WM is needed in chess imagery (Robbins et al., 1996). This is in line with results showing that executive resources are needed for the constructive mental synthesis of multiple parts and for the conscious experience of imagery (Pearson, De Beni, & Cornoldi, 2001). For chess experts, the active imagery in WM has been conceptualised in the memory literature as the minds eye (Chase & Simon, 1973). Campitelli and Gobet (2005) have proposed that the chess experiments of Studies I and II indicate activation in the minds eye where the perceptual chunking process proceeds as it does with perceptual stimuli. This suggestion implies that many items have to be maintained in WM during the presentation of stimuli, and that only when all or most of the items have been presented does perceptual pattern matching become possible. However, if the capacity of WM is as small as two chunks, as has been suggested for chess (Gobet & Clarkson, 2004) and if the visual images fade quickly, as the main theories argue (Kosslyn, 1994; Logie, 1995), the suggestion that results of our blindfold chess experiments can be interpreted with perceptual chunking in the minds eye is implausible. How could direct perceptual pattern matching occur if there are no patterns in the minds eye before several items have been presented? And how could all these items be kept active in the limited WM system until there were enough items for pattern matching? Thus, the explanation offered by Campitelli and Gobet (2005) seems unlikely because of the small capacity of WM for unchunked items. It is more likely that when there is an overload of information, one must choose what to include in a mental image (Rouw, Kosslyn, & Hamel, 1997). The chess players possibly took note of a transformation in a position only if it occurred in the functional figure rather than in the functional background. Thus, it is unlikely that imagery causes activation in certain areas of the minds eye that then leads to automatic perceptual pattern matching. Rather, imagery may facilitate a top-down process where conceptual chunking is used for figure-ground organisation (cf., Heil, Rsler, & Henninghausen, 1993). Presumably, visuo-spatial WM had a role when the taxi drivers in Study III rehearsed street names by creating visual images from simple verbal descriptions, such as individual street names. Similar phenomena have been demonstrated in research on how language is transformed into spatial cognition and is used for navigation in a spatial environment (eg., Denis, Daniel, & Fontaine, 2001). Furthermore, the taxi drivers also reported using mental driving in the city as a mnemonic for the street names. It is possible that visuo-spatial WM has a role in imagining driving, just like it plays a crucial role in supporting mental actions in encoding verb phrases (Engelkamp & Zimmer, 1994; Logie, Engelkamp, Dehn, & Rudkin, 2001).

75

Apparently, WM had a role when musicians constructed auditory images from the successive presentation of visual notes. By constructing auditory images in their inner ear they were able to create meaningful structures and thus outperformed the control group. The review of Study V also presented several examples of musical images that are processed in working memory. The research reviewed suggests that acoustic imagery is involved in pitch comparison and melody tasks, whereas subvocal rehearsal is only involved in melody tasks (Keller, Cowan, & Saults, 1995; Logie & Edworthy, 1986; Pechmann & Mohr, 1992). The second finding of the review was that there are multiple auditory components in music. For example, such temporal information as rhythm, tempo, and total duration are in some cases encoded jointly with nontemporal information like pitch intervals (Carroll-Phelan & Hampson, 1996; Peretz & Kolinsky, 1993). However, if the two dimensions are structurally incompatible or if the individuals degree of musical experience is low, temporal and nontemporal dimensions may require independent processing (Boltz, 1998). An interesting question presented in Study V was, how pitch and rhythm are encoded in musical imagery and what working memory sub-components are involved. It is likely that other modalities in addition to the auditory one can be used when a mental representation is constructed from visual notation. Although imagery representations often seem to be sense-specific, it is also possible to use several modalities at a time, as is the case in perceptual tasks (Intons-Peterson, 1992). The first pilot experiment of Study IV suggested that visual and other non-musical factors enhanced the construction of representations from visual notes. This corresponds to previous research showing that the visual complexity of briefly presented musical patterns and gestalt properties of visual images influence recall (Halpern & Bower, 1982; Saariluoma, 1992a; see also Palmer, 1977). Furthermore, as reviewed in Study V, previous research suggests that representation construction in music may involve several strategies in rehearsing note sequences mentally. Besides verbal strategies, the participants in Study IV visualised the location of notes on their instruments or on a stave and used motor imagery of how the note sequences would be performed on their instruments. Likewise, Mikumo (1994) has shown that finger tapping melodies as if playing a mute piano improves recall of musical patterns. This suggests that motor imagery plays a role in musical representations, and it is possible that the visuo-spatial working memory has a role in musical imagery similar to that in mental actions (cf., Logie, Engelkamp, Dehn, & Rudkin, 2001). Study V suggested that musical imagery not only stands at the intersection of memory and perception, but also at the intersection of several sense modalities; musical images are possibly processed in a working

76

memory sub-component that was not defined in the traditional WM models. This suggestion is in line with the episodic buffer of Baddeleys working memory model that has means of integrating several types of information and of storing complex images, a task in which the contribution of LTM is important (Baddeley, 2000). In sum, the present thesis shows that experts images are constructed in WM, and that experts are able to circumvent the limits of WM by also using LTM knowledge and skills. Experts images often seem to be complex and multimodal. Thus, they may rely on several WM subcomponents and they require executive resources.

9.5. No end to the imagery debate

Even though a substantial amount of behavioural and brain research data suggests that imagery and perception share some common underlying mechanisms (Kosslyn, 1994, 2005), theorists do not agree about whether the perception likeness of mental imagery requires a distinct depictive representational format in the cognitive architecture (Pylyshyn, 2002). It has been suggested that, despite our intuitive and subjective experience, the concept of mental imagery is not fundamental in the cognitive sciences (Pylyshyn, 1973, 1981, 2002). If this is accepted, it would mean that mental imagery is an epiphenomenal and intuitive construct that can be reduced to an abstract propositional code that underlies all information representation in the cognitive system (Pylyshyn, 2002). Furthermore, even if the mechanism underlying mental imagery and perception were the same, it does not require that the elements of mental images or perception are depictive in a sense that they resemble the physical environment (Pylyshyn, 2002). From that perspective, mental images are like any cognitive representations, and mechanisms related to memory are involved rather than the mechanisms of perception. In the present studies the evidence for mental imagery is based on subjective interpretations and indirect findings that are difficult to interpret without assuming mental imagery. Although experts in the present studies report the subjective experience of images that resemble the physical world, the present thesis did not directly address the issue of depictive resemblance of images to the physical world. Thus, the results do not provide evidence for the assumption that experts imagery representations were depictive in the sense that they resembled the physical environment. It is also obvious from some brain studies of expert imagery that, although primary sensory areas are sometimes involved in mental imagery, they do not seem to be engaged in all imagery tasks. For example, brain activation in chess players (studied with positron emission tomography)

77

during usage of visuo-spatial representations in blindfold chess did not show any activity in the primary visual areas when the information was presented auditorily (Saariluoma, Karlsson, Lyytinen, Ters, & Geisler, 2004). However, Kosslyn (1994; Kosslyn, Thompson, Kim, & Alpert, 1995) has suggested that distinct areas of visual cortex are involved in images at different levels of scale, and thus activity in the visual buffer or WM in imagery tasks is not simply activity in the topographically organised areas of visual cortex. Other brain areas are also involved when mental images are constructed in WM or in the visual buffer (Kosslyn, 1994). Thus, although brain activation in the areas involved in perception suggests that images are used, the lack of activation in the sensory areas may indicate that images are processed at a higher cognitive level. Although the same presentation method was used in the present studies in different domains, this does not necessarily result in similar imagery representations. In chess, fragments of chess board result a simultaneous visuo-spatial representation, whereas representations of taxi routes and melodies comprise sequential elements in visual and auditory modalities, respectively. Therefore, even though the method of sequential presentation allows some comparability between the domains, it is possible that the method nevertheless leads to different processes and representations in different domains, and that the results do not reflect imagery of a same kind. At present, only the subjective experience of imagining connects a wide variety of studies on mental imagery, and it is obvious that the current theories are not able to connect all of the data. Apparently, some of the disagreements concerning the nature of mental imagery may reflect the need for re-defining of the concept of mental imagery. The concept of mental imagery is used in the context of basic sensory features of mental images (Ishai & Sagi, 1995), in using images in demanding cognitive tasks such as language comprehension and problem solving (Denis, 1991), and as here, in the context of images that only experts are able to construct in their domain of expertise. This situation involves similar conceptual problems that have been discussed in WM research where several perspectives are used, such as the approaches that study basic capacity, complex skills, and expert performance (Ericsson & Delaney, 1999). The situation in imagery research, however, is even more complex than the situation in working memory research, since it is not only the level of skill required that differs between studies; the tasks also reflect different positions in the continuum from perception to memory. Mental images have been proposed to be neither pure descriptions nor pure mental pictures; both symbolic and perceptual properties are said to be involved along with a continuum from perceptual to conceptual images (Kaufmann, 1996; Marks, 1983). It is also likely that, as Reisberg proposes (in Logie, Cornoldi, Brandimonte, Kaufmann, & Reisberg, 1996, p. 175), depictive and

78

descriptive properties are not separable but are firmly integrated as a package and cannot be pulled from each other. The knowledge-weighted model of mental imagery proposes that images of familiar objects or tasks may differ from those related to unfamiliar tasks; and, at least in some cases, imagery involves memory with specific properties rather than a distinct and separate process (Intons-Peterson, 1996). Also, results of the present studies may indicate that in skilled imagery, mechanisms related to memory are involved rather than perception (see also Kolers & Smythe, 1979). The subsequent problem is whether the results of the present studies truly contribute to the imagery research or only to research on expert memory? This problem raises further questions 1 : Do experts have better general imagery than others? Or is experts' imagery more efficient only in their own domain? The well-documented domain specificity of skill effects implies that experts do not perform better than novices in general psychometric tests. It has also been shown that chess players do not have better visuospatial abilities than others, and thus their exceptionality in domain-specific visuo-spatial chess memory tasks is not related to better imagery ability per se (Waters, Gobet, & Leyden, 2002). On the other hand, there is also evidence that experts may have better imagery abilities than others. For example, Dror, Kosslyn and Waag (1993) showed that in some mental imagery tasks, such as judging metric spatial relations or mentally rotating objects, pilots showed better visuo-spatial abilities than control participants. However, this result does not necessarily imply that experts had better imagery abilities; their imagery may be qualitatively different and rely more on automatic processes (Reed, 2002). Nevertheless, the issue of individual differences in imagery (Cornoldi & Vecchi, 2003; Kozhevnikov, Kosslyn, & Shephard, 2005) needs further studies in the context of experts imagery. The results of the present thesis show that experts representations are more efficient than novices. Although many expertise effects may be related to automatic and direct pattern matching, the method of the present studies requires that experts construct images with active and controlled processes; they also report the subjective experience of imagining. Thus, when experts construct images, they rely on modality-specific working memory components. What is special to experts images is that they are able to circumvent the limits of WM by employing their LTM knowledge and skills. Experts are not better than novices in using mental imagery per se (not do they have better WM per se); they are simply able to overcome the limits of

This is one of the issues raised by Prof. Cesare Cornoldi in his pre-examination of this thesis. For a review of visual mental imagery and individual differences see Cornoldi and Vecchi (2003)

79

their imagery system (just as they are able to overcome the limits of WM in general). Our results also show that experts images are conceptually organised and interpreted rather than merely depictive. Experts exceptionality in constructing mental images (and in working memory tasks), is restricted to domain specific tasks. In skilled imagery, interaction between WM and LTM has a bigger role than in some other imagery tasks, as was noted above. The present study has focused on the format of mental representations that experts use, and on the capacity limitations of the mental imagery system. The capacity-focused approach has been criticised for not being able to consider the content of mental representations and how information is selected for construction into a mental representation (Saariluoma, 1995, 1997). The concept of apperception offers a way of addressing the issue of how experts construct representations and select the contents for them (Saariluoma, 1992b, 1995, 1997). The distinction between the content, what is represented, and the format - how it is represented - has recently also been raised among imagery researchers (Rouw, Kosslyn, & Hamel, 1997). It has been suggested that the contents of images and percepts differ for several reasons that are related to the source of information and the capacity limitations of the imagery system (Rouw, Kosslyn, & Hamel, 1997). To sum up, the main opponents of the imagery debate seem to agree only on two issues: firstly, that most people have mental images, that is, subjective experiences of imaging that resemble the experience of perceiving (Pylyshyn, 2002), and secondly, that knowledge can influence imagery (Kosslyn, Thompson, Ganis, 2002). The present study shows that these issues also apply to skilled images: experts report a subjective experience of imagining, and the role of pre-learned knowledge and knowledge-based interpretation is evident in experts imagery. The present research also shows that the expertise advantage found in tasks of constructing images is related to conceptual chunking rather than to perceptual pattern matching mechanisms. Furthermore, skilled imagery does not require that experts have exceptional imagery abilities per se; rather, their chunking of information in working memory is superior. Therefore, they have outstanding processing of domain specific images in WM and, thus, they perform better than novices in memory and problem solving tasks that require mental imagery. The present thesis suggests that, to resolve the debate on the nature of mental imagery, future research must consider such factors as the level of skill required in the task and the position of the task in the continuum from perception to memory (see also Palmer, 1977). The concept of mental imagery needs a re-definition that takes into account the variety of imagery research and the new evidence for skilled images.

80

9.6. Future issues for research on expert imagery

Experimental findings in expert memory research have greatly influenced memory research, and it is evident that findings from future studies of experts images will also contribute to the literature on memory and mental imagery. The present research introduces a new phenomenon of skill effects on the incremental construction of mental representations that the theories on expert memory, mental imagery, and WM should be able to address. The method developed in the present studies is a good starting point for future studies that combine these research approaches. Nevertheless, the method also has limitations that havent yet been discussed here. One is that it does not necessarily reveal the genuine excellence the experts possess. Not all chess players are used to playing blindfold chess; the task of taxi drivers is to actually drive in the city rather than recall lists of street names; and, although musicians have practiced associating visual notation and auditory melodies, the method of briefly presenting notation, or presenting it one note at a time, is not a normal task for musicians. Therefore, the present method is too limited to provide insights into expert performance in the complex task environments in specific domains. Therefore, more ecologically valid presentation methods are needed if the main interest of the research is related to aspects other than the mechanisms underlying representation construction. On the other hand, our research approach is compatible with some recent, fruitful and timely research topics. The multi-modal character of working memory representations is a little-examined phenomenon that can be captured with the method used for the present study. This question is relevant especially in music, where expert musicians deal with visually notated patterns, auditory tones, and motor representations of performing the melody. A subsequent research question is to determine how multimodal information is maintained and rehearsed in working memory, and how information from perceptual processes, several working memory subcomponents, and long-term memory are attended to. The method of capturing incremental construction of mental images is one way to gather more data on the still under-researched issue of how information from the phonological loop, the visuo-spatial subcomponent, and LTM, are held in WM in some integrated form; that is, in the episodic buffer proposed by Baddeley (2000). The present results are compatible with the description of the episodic buffer as a system related to integrative processes, such as (conceptual) chunking, and as a seat of creating new cognitive representations. However, the recent continuity model of working memory (Cornoldi & Vecchi, 2003) proposes an alternative, fruitful approach where working memory processes are defined on two continuums: the different

81

types of material processed and the different levels of active elaboration and integration. An especially timely topic in memory research is the nature of the interaction between long-term and short-term memory. Outside of expertise literature, this topic has been studied mainly in the 1960s and, again recently, as the effect of background knowledge on immediate serial recall (Botvinick, 2005; Botvinick & Bylsma, 2005). Nevertheless , this issue has not been solved by most models of serial recall (Botvinick, 2005; Henson, 1998). The nature of interaction between long-term and short-term memory is also central in the models of working memory. The idea of a close relationship between WM and LTM is presented in Cowans (1995) theory where WM is an activated part of LTM. As noted above, in the renewed WM model of Baddeley (2000) a new separate component, the episodic buffer, is proposed as the seat of the cooperation between WM and LTM. Recently, theoretical discussion has taken place concerning whether the short-term and long-term memory systems use similar mechanisms, as opposed to being separate systems (Nairne, 2002; Ruchkin, Grafman, Cameron, & Berndt, 2003). Since the incremental construction of mental images is a task in which LTM and STM systems interact, it offers a fruitful method for raising new questions about the timely topic of the cooperation between STM and LTM systems. The present studies show that when experts construct mental representations, LTM knowledge and stimulus features are combined in WM in a non-trivial manner. Mental representations do not include all the details of the presented stimuli; however, they also include features that were not present in the stimuli at all. The question, then, is how LTM knowledge and stimuli combine into a mental representation in working memory? Within STM, as soon as the items have been presented for immediate serial recall, the formed memory trace begins to degrade due to decay or interference (Baddeley, Thomson, & Buchanan, 1975; Nairne, 1990, 2002). The degradation of the STM trace can be hindered by a reintegration process in which the degraded trace is reconstructed for recall (Hulme et al., 1997; Lewandowsky, 1999; Nairne, 1990; Schweickert, 1993). The reconstruction of a STM trace involves LTM-related factors. Schweickert (1999) stresses that the probability a degraded trace for the item is correctly reconstructed reflects a contribution of LTM - not in the sense that the item is recalled from LTM, but in the sense that knowledge of the domain is used in the reconstruction. In the verbal domain, for example, word frequency and lexicality affect the ability to reconstruct items in STM (Hulme et al., 1997; Schweickert, Chen, & Poirier, 1999; Thorn, Gathercole, & Frankish, 2005). Furthermore, Stuart and Hulme (2000) reported that familiarisation with verbal stimulus material improved the serial recall of low-frequency

82

words to the level of high-frequency words, which indicates that inter-item associations in LTM affect STM performance. The future studies can ask, for example, are experts only superior in redintegrating whole lists and patterns in WM, or are they also better at maintaining the degraded memory traces in STM? In the other words, does domain-specific LTM knowledge only facilitate the perfect recall and redintegration of lists in WM, or does it also support storage of degraded traces (cf., Thorn, Gathercole, & Frankish, 2005). An additional issue is how modality-dependent and modality-independent features suggested in the feature model of Nairne (1990) contribute to experts representations and images in WM. Detailed error classifications and analyses of serial position effects could provide specific information about whether the WM representations of experts differ from those of novices. For research on expert memory, the mechanisms underlying the recall of entire lists and the nature of the degradation of the lists in memory is a more fundamental issue than the recall of individual items, since the core of experts exceptional memory performance is the grouping of individual items into chunks, rather than memory for individual items per se. In contrast to most studies of STM in which element ordering in lists is meaningless, research on expert memory includes examples of domains where the sequential structure of material is inherently relevant. Digit span experts use knowledge of running times for the encoding of lists of digits (Ericsson, 1985), taxi drivers exceptional memory for lists of street names is based on knowledge of the sequential order of the streets in the city, and musicians superior memory for notes requires that items are ordered by the rules of music (Sloboda, 1984, 1985). Along with research on the bigram frequency effect from the 1960s, until recently there has been only two studies on the role of background knowledge on the recall of entire lists (Study IV; Baddeley & Wilson, 2002) that employed material drawn from a domain where the structure of sequences is relevant (Botvinick, 2005). The only systematic example involving the effect of background knowledge on the recall of an entire list are the recent studies by Botvinick and Bylsma (2005). They used artificial grammar containing lists where the probability of a sequential structure of the same six items varied. The results showed that the stimulus probability affected accuracy and the nature of recall errors, suggesting that the effect of knowledge concerning the sequential structure affects immediate serial recall at the level of entire lists, not only at the level of individual item (Botvinick, 2005). Botvinick and Bylsma (2005) showed that the list structure rather than individual items are learned. This notion is related to the broader issue of what experts acquire through training, a question that also has implications for applied cognitive psychology (Ericsson, 2005) and that could be studied in more detail using the method of the present studies.

83

Expert imagery is a complex issue where several concepts and theories meet and is in line with the recent trend of combining different lines of research, such as STM capacity and expert behaviour (Gobet & Clarkson, 2004). Various imagery researchers have also discussed the overlap of the concepts emerging from separate research areas. They suggest a correspondence between mental imagery and WM representations (Kosslyn, 1994; Logie, 1995) and propose that the minds eye in the chunking and template theories play the same role as the visual buffer in Kosslyns (1994) theory of imagery and that chunks and templates correspond to structural descriptions in Kosslyns model (Campitelli & Gobet, 2005). In the future, this discussion between detached lines of research will certainly lead to better understanding of the complex issue of expert imagery and cognition. The present thesis shows that experts are able to adapt to novel tasks and to use their knowledge and skills to outperform novices. In future studies, a fruitful approach to expert representation construction would be to combine the complex tasks that only experts can perform with experimental methods that are capable of capturing the detailed mechanisms that research on STM and WM has identified. In the future, the results of this research approach may also help to understand how to facilitate the use of pre-learned knowledge and skills when facing a novel task. For a beginner, most tasks are novel and difficult, and it is critical to be able to proceed even though the task is not trivial and pre-learned knowledge cannot be automatically used. With an understanding of how experts are able to use pre-learned knowledge in deliberate and conscious ways, ways could be found of teaching these methods to beginners, thus increasing the knowledge of the cognitive mechanisms involved in learning and developing expertise.

84

10. Constructing skilled images

The exceptional cognitive performance of experts is one of the main puzzles of cognitive psychology. Despite having the normal, limited cognitive system, experts are able to outperform novices in domain-relevant tasks. They are able to exceed the limits of WM by using the knowledge and skills of LTM because their representations consist of chunks of relevant information rather than just a few independent items. The results in three domains, chess, taxi driving and music, showed that experts are able to use pre-learned knowledge and skills even in novel imagery tasks that required incremental construction of mental representations. Furthermore, the results indicate that, even though the task in the present studies required perception-like mental representations, the mechanisms of conceptual chunking rather than of automatic perceptual pattern matching underlie expert performance. The results suggest that experts images are highly organised and that they do not represent all the given information as a true analogue record; functionally relevant information is better represented than irrelevant features. This thesis demonstrates that expert imagery is a complex issue where several key concepts of cognitive psychology meet; for example, working memory, imagery, and expertise. The research method introduced in the present studies proved reliable. In the future it can be applied to the study of several important topics, such as how information from separate WM subcomponents and LTM can be united into a meaningful episode, as is the case with complex images. The findings of the present thesis on expert imagery and the recent findings of research on STM and WM converge to a new approach for studying the cooperation between WM and LTM.

85

References
Adelson, B. (1984). When novices surpass experts: The difficulty of a task may increase with expertise. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10(3), 483-495. Allard, F., & Starkes, J. L. (1991). Motor-skill experts in sports, dance, and other domains. In K. A. Ericsson & J. Smith (Eds.), Toward a general theory of expertise. Prospects and limits. Cambridge: Cambridge University Press. Anderson, J. R. (1983). The architecture of cognition. Cambridge, Mass.: Harvard University Press. Anderson, R. E., & Helstrup, T. (1993). Visual discovery in mind and on paper. Memory & Cognition, 21(3), 283-293. Anzai, Y. (1991). Learning and use of representations for physics expertise. In K. A. Ericsson & J. D. Smith (Eds.), Toward a general theory of expertise. Prospects and limits. Cambridge: Cambridge University Press. Attneave, F., & Curlee, T. (1983). Locational representation in imagery: a moving spot task. Journal of Experimental Psychology: Human Perception & Performance 9, 20-30. Bachmann, T., & Oit, M. (1992). Stroop-like interference in chess players' imagery: an unexplored possibility to be revealed by the adopted moving-spot task. Psychological Research, 54, 27-31. Baddeley, A. D. (1986). Working memory. Oxford: Clarendon Press. Baddeley, A. D. (1996). The fractionation of working memory. Proceedings of the National Academy of Sciences, 93, 13468-13472. Baddeley, A. D. (2000). The episodic buffer: a new component of working memory? Trends in Cognitive Sciences, 4(11), 417-423. Baddeley, A. D., & Hitch, G. (1974). Working memory. In G. Bower (Ed.), The psychology of learning and motivation (Vol. 8, pp. 47-89). New York: Academic Press. Baddeley, A. D., & Lewis, V. (1981). Inner active processes in reading: The inner voice, the inner ear, and the inner eye. In A. M. Lesgold & C. A. Perfetti (Eds.), Interactive processes in reading (pp. 107-129). Hillsdale, N.J.: Lawrence Erlbaum Associates. Baddeley, A. D., & Logie, R. H. (1992). Auditory imagery and working memory. In D. Reisberg (Ed.), Auditory imagery (pp. 179-197). Hillsdale, New Jersey: Laurence Erlbaum Associates. Baddeley, A. D., Thomson, N., & Buchanan, M. (1975). Word length and the structure of short-term memory. Journal of Verbal Learning and Verbal Behavior, 14, 575-589. Baddeley, A. D., & Wilson, B. A. (2002). Proce recall and amnesia: implications for the structure of working memory. Neuropsychologia, 40, 1737-1743. Ball, L. J., Evans, J. S. B. T., Dennis, I., & Ormerod, T. C. (1997). Problem-solving strategies and expertise in engineering design. Thinking and Reasoning, 3(4), 247-270. Bensafi, M., Porter, J., Pouliot, S., Mainland, J., Johnson, B., Zelano, C., et al. (2003). Olfactory activity during imagery mimics that during perception. Nature Neurosciences, 6(11), 1142-1144. Billman, D., & Shaman, D. (1990). Strategy knowledge and strategy change in skilled performance: A study of the game Othello. American Journal of Psychology, 103(2), 145-166. Binet, A. (1893/1966). Mnemonic virtuosity: A study of chess players. Genetic Psychology Monographs, 74, 127-162. Boltz, M. G. (1998). The processing of temporal and nontemporal information in the remembering of event durations and musical structure. Journal of Experimental Psychology: Human Perception and Performance, 24(4), 1087-1104. Botvinick, M. M. (2005). Effects of domain-specific knowledge on memory for serial order. Cognition, 97(2), 135-151.

86

Botvinick, M. M., & Bylsma, L. M. (2005). Regularization in short-term memory for serial order. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(2), 351-358. Bower, G. H. (1970). Imagery as a relational organizer in associative learning. Journal of Verbal Learning and Verbal Behavior, 9, 529-533. Brandimonte, M. A., Hitch, G. J., & Bishop, D. (1992a). Influence of short-term memory codes on visual image processing: Evidence from image transformation tasks. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 157-165. Brandimonte, M. A., Hitch, G. J., & Bishop, D. (1992b). Verbal recoding of visual stimuli impairs mental image transformations. Memory & Cognition, 20, 449-455. Brodsky, W., Henik, A., Rubinstein, B.-S., & Zorman, M. (2003). Auditory imagery from musical notation in expert musicians. Perception & Psychophysics, 65(4), 602-612. Campitelli, G., & Gobet, F. (2005). The mind's eye in blindfold chess. European Journal of Psychology, 17(1), 23-45. Carroll-Phelan, B., & Hampson, P. J. (1996). Multiple components of the perception of musical sequences: A cognitive neuroscience analysis and some implications for auditory imagery. Music Perception, 13(4), 517-561. Chambers, D., & Reisberg, D. (1985). Can mental images be ambiguous? Journal of Experimental Psychology: Human Perception and Performance, 11, 317-328. Chambers, D., & Reisberg, D. (1992). What an image depicts depends on what an image means. Cognitive Psychology, 24, 145-174. Charness, N. (1976). Memory for chess positions: Resistance to interference. Journal of Experimental Psychology: Human Learning and Memory, 2, 641-653. Charness, N. (1988). Expertise in chess, music, and physics: A cognitive perspective. In L. K. Obler & D. Fein (Eds.), The exceptional brain. Neuropsychology of talent and special abilities. New York: The Guilford Press. Charness, N. (1992). The impact of chess research on cognitive science. Psychological Research, 54, 4-9. Chase, W. G. (1983). Spatial representations of taxi drivers. In D. A. Rogers & J. A. Sloboda (Eds.), The acquisition of symbolic skills (pp. 391-405). New York: Plenum Press. Chase, W. G., & Ericsson, K. A. (1981). Skilled Memory. In J. R. Anderson (Ed.), Cognitive skills and their acquisition. Hillsdale, NJ: Lawrence Erlbaum Associates. Chase, W. G., & Ericsson, K. A. (1982). Skill and working memory. The Psychology of Learning and Motivation, 16, 1-58. Chase, W. G., & Simon, H. A. (1973). The mind's eye in chess. In W. G. Chase (Ed.), Visual information processing (pp. 215-281). New York: Academic Press. Chi, M. T. H., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R. J. Sternberg (Ed.), Advances in the psychology of human intelligence (Vol. 1). Hillsdale, NJ: Lawrence Erlbaum Associates. Clifton, J. V. (1986). Cognitive components in music reading and sight reading performance. Unpublished Doctoral dissertation, University of Waterloo, Waterloo, Ontario. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, New Jersey: Lawrence Erlbaum Associates. Cooke, N. J., Atlas, R. S., Lane, D., M., & Berger, R. C. (1993). Role of high-level knowledge in memory for chess positions. American Journal of Psychology, 106(3), 321-351. Cornoldi, C., & Vecchi, T. (2003). Visuo-spatial working memory and individual differences. Hove, East Sussex: Psychology Press. Cowan, N. (1995). Attention and memory: An integrated framework. New York: Oxford University Press. Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral & Brain Sciences, 24(1), 87-185. Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671-684.

87

Crowder, R. G. (1989). Imagery for musical timbre. Journal of Experimental Psychology: Human Perception and Performance, 15(3), 472-478. Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19, 450-466. Deakin, J. M., & Allard, F. (1991). Skilled memory in expert figure skaters. Memory & Cognition, 19, 79-86. Denis, M. (1991). Image and cognition. Hertfordshire: Harvester Wheatsheaf. Denis, M., & Cocude, M. (1989). Scanning visual images generated from verbal descriptions. European Journal of Cognitive Psychology, 1(4), 293-307. Denis, M., & Cocude, M. (1992). Structural properties of visual images constructed from poorly or well-structured verbal descriptions. Memory & Cognition, 20, 497-506. Denis, M., & Cocude, M. (1997). On the metric properties of visual images generated form verbal descriptions: Evidence for the robustness of the mental scanning effect. European Journal of Cognitive Psychology, 9(4), 353-379. Denis, M., Daniel, M.-P., & Fontaine, S. (2001). Language, spatial cognition, and navigation. In M. Denis, R. H. Logie, C. Cornoldi, M. De Vega & J. Engelkamp (Eds.), Imagery, Language and Visuo-spatial thinking (pp. 137-160). Hove: Psychology Press. Denis, M., & Zimmer, H. D. (1992). Analog properties of cognitive maps constructed from verbal descriptions. Psychological Research, 54, 286-298. Dror, I. E., Kosslyn, S. M., & Waag, W. L. (1993). Visual-spatial abilites of pilots. Journal of Applied Psychology, 78(5), 763-773. Elo, A. (1978). The ratings of chess players: past and present. London: Batsford. Engelkamp, J., & Zimmer, H. D. (1994). The human memory: A multimodal approach. Gttingen: Hogrefe & Huber Publishers. Engle, R. W., Tuholski, S. W., Laughlin, J. E., & Conway, A. R. A. (1999). Working memory, short-term memory, and general fluid intelligence: A latent variable approach. Journal of Experimental Psychology: General, 128, 309-331. Ericsson, K. A. (1985). Memory skill. Canadian Journal of Psychology, 39, 188-231. Ericsson, K. A. (1988). Analysis of memory performance in terms of memory skill. In R. J. Sternberg (Ed.), Advances in the psychology of human intelligence (Vol. 4, pp. 137-179). Hillsdale, NJ: Erlbaum. Ericsson, K. A. (2005). Recent advances in expertise research: A commentary on the contributions to the special issus. Applied Cognitive Psychology, 19, 233-241. Ericsson, K. A., & Charness, N. (1994). Expert performance. Its structure and acquisition. American Psychologist, 49(8), 725-747. Ericsson, K. A., & Delaney, P. F. (1999). Long-term working memory as an alternative to capacity models of working memory in everyday skilled performance. In A. Miyake & P. Shah (Eds.), Models of working memory. Mechanisms of active maintenance and excecutive control (pp. 257-297). Cambridge: Cambridge University Press. Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102(2), 211-245. Ericsson, K. A., & Kintsch, W. (2000). Shortcomings of generic retrieval structures with slots of the type that Gobet (1993) proposed and modelled. British Journal of Psychology, 91, 571-590. Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional performance: Evidence of maximal adaptation to task constraints. Annual Review of Psychology, 47, 273-305. Ericsson, K. A., Patel, V., & Kintsch, W. (2000). How experts' adaptation to representative task demands account for the expertise effect in memory recall: Comments on Vicente and Wang (1998). Psychological Review, 107(3), 578-592. Ericsson, K. A., & Polson, P. G. (1988). An experimental analysis of the mechanisms of a memory skill. Journal of Experimental Psychology: Learning,Memory, and Cognition, 14(2), 305-316.

88

Ericsson, K. A., & Staszewski, J. J. (1989). Skilled memory and expertise: Mechanisms of exceptional performance. In D. Klahr & K. Kotovsky (Eds.), Complex information processing: The impact of Herbert A. Simon (Vol. 2, pp. 235-267). New York: Wiley. Farah, M., J., & Smith, A., F. (1983). Perceptual interference and facilitation with auditory imagery. Perception & Psychophysics, 33(5), 475-478. Finke, R. (1985). Theories relating mental imagery to perception. Psychological Bulletin, 98, 236-259. Finke, R., Pinker, S., & Farah, M., J. (1989). Reinterpreting visual patterns in mental imagery. Cognitive Science, 13, 51-78. Finke, R., & Slayton, K. (1988). Explorations of creative visual synthesis in mental imagery. Memory & Cognition, 16(3), 252-257. Frey, P. W., & Adesman, P. (1976). Recall memory for visually presented chess positions. Memory & Cognition, 4, 541-547. Freyhof, H., Gruber, H., & Ziegler, A. (1992). Expertise and hierarchical knowledge representation in chess. Psychological Research, 54, 32-37. Gathercole, S. E. (1995). Is nonword repetition a test of phonological memory or long-term knowledge? It all depends on the nonwords. Memory & Cognition, 23(1), 83-94. Gathercole, S. E., Willis, C. S., Emslie, H., & Baddeley, A. D. (1991). The influences of number of syllables and word-likeness on children's repetition of nonwords. Applied Psycholinguistics, 12, 349-367. Gobet, F. (1998). Expert memory: a comparison of four theories. Cognition, 66, 115-152. Gobet, F. (2000a). Retrieval structures and schemata: A brief reply to Ericsson and Kintsch. British Journal of Psychology, 91(591-594). Gobet, F. (2000b). Some shortcomings of long-term working memory. British Journal of Psychology, 91, 551-570. Gobet, F., & Clarkson, G. (2004). Chunks in expert memory: Evidence for the magical number four... or is it two? Memory, 12(6), 732-747. Gobet, F., Lane, P. C. R., Croker, S., Cheng, P. C.-H., Jones, G., Oliver, I., et al. (2001). Chunking mechanisms in human learning. Trends in Cognitive Sciences, 5(6), 236-243. Gobet, F., & Simon, H. A. (1996a). Recall of random and distorted chess positions: Implications for the theory of expertise. Memory & Cognition, 24(4), 493-503. Gobet, F., & Simon, H. A. (1996b). Templates in chess memory: A mechanism for recalling several boards. Cognitive Psychology, 31, 1-40. Gobet, F., & Simon, H. A. (2000). Five seconds or sixty? Presentation time in expert memory. Cognitive Science, 24(4), 651-682. Gobet, F., & Waters, A. J. (2003). The role of constraints in expert memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29(6), 1082-1094. Goldin, S. E. (1978). Effects of orienting tasks on recognition of chess positions. American Journal of Psychology, 91(4), 659-671. de Groot, A. D. (1965). Thought and choice in chess. The Hague: Mouton. de Groot, A. D. (1966). Perception and memory versus thought: Some old ideas and recent findings. In B. Kleinmuntz (Ed.), Problem solving. New York: Wiley. Halpern, A. R. (1988). Mental scanning in auditory imagery for songs. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14(3), 434-443. Halpern, A. R., & Bower, G. H. (1982). Musical expertise and melodic structure in memory for musical notation. American Journal of Psychology, 95(1), 31-50. Hatano, G., & Osawa, K. (1983). Digit memory of grand experts in abacus-derived mental calculation. Cognition, 15, 95-110. Hatta, T., Hirose, T., Ikeda, K., & Fukuhara, H. (1989). Digit memory of soroban experts: Evidence of utilization of mental imagery. Applied Cognitive Psychology, 3(1), 23-33. Heil, M., Rsler, F., & Henninghausen, E. (1993). Imagery-perception interaction depends on the shape of the image: A reply to Farah (1989). Journal of Experimental Psychology: Human Perception and Performance, 19(6), 1313-1320.

89

Henson, R. N. A. (1998). Short-term memory for serial order: the start-end model. Cognitive Psychology, 36, 73-137. Hishitani, S. (1989). The usefulness of studies of imagery experts in imagery research. Journal of Mental Imagery, 13(2), 119-134. Hishitani, S. (1990). Imagery experts: How do expert abacus operators process imagery? Applied Cognitive Psychology, 4(1), 33-46. Hitch, G. J., Brandimonte, M. A., & Walker, P. (1995). Two types of representation in visual memory: Evidence from the effects of stimulus contrast on image combination. Memory & Cognition, 23(2), 147-154. Holding, D. H. (1985). The psychology of chess skill. Hillsdale, NJ: Lawrence Erlbaum Associates. Holding, D. H. (1992). Theories of chess skill. Psychological Research, 54, 10-16. Holding, D. H., & Reynolds, R. I. (1982). Recall or evaluation of chess positions as determinants of chess skill. Memory & Cognition, 10(3), 237-242. Hubbard, T., L., & Stoeckig, K. (1988). Musical imagery: Generation of tones and chords. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14(4), 656-667. Hulme, C., Roodenrys, S., Schweickert, R., Brown, G. D. A., Martin, S., & Stuart, G. (1997). Word-frequency effects on short-term memory tasks: evidence for a redintegration process in immediate serial recall. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23(5), 1217-1232. Intons-Peterson, M. J. (1992). Components of auditory imagery. In D. Reisberg (Ed.), Auditory imagery (pp. 45-71). Hillsdale, New Jersey: Lawrence Erlbaum Associates. Intons-Peterson, M. J. (1996). Integrating the components of imagery. In M. De Vega, M. J. Intons-Peterson, P. N. Johnson-Laird, M. Denis & M. Marschark (Eds.), Models of visuospatial cognition. New York: Oxford University Press. Intons-Peterson, M. J., & McDaniel, M. A. (1991). Symmetries and asymmetries between imagery and perception. In C. Cornoldi & M. A. McDaniel (Eds.), Imagery and cognition (pp. 47-77). New York: Springer-Verlag. Ishai, A., & Sagi, D. (1995). Common mechanisms of visual imagery and perception. Science, 268(23 June), 1772-1774. Kaufmann, G. (1996). The many faces of mental images. In C. Cornoldi, R. H. Logie, M. A. Brandimonte, G. Kaufmann & D. Reisberg (Eds.), Stretching the imagination. Representation and transformation in mental imagery. (pp. 77-118). New York: Oxford University Press. Keller, T., A., Cowan, N., & Saults, J. S. (1995). Can auditory memory for tone pitch be rehearsed? Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(3), 635-645. Kolers, P. A., & Smythe, W. A. (1979). Images, symbols, and skills. Canadian Journal of Psychology, 33(3), 158-184. Kosslyn, S. M. (1975). Information representation in visual images. Cognitive Psychology, 7, 341-370. Kosslyn, S. M. (1980). Image and mind. Cambridge, MA: Harvard University Press. Kosslyn, S. M. (1994). Image and brain: The resolution of the imagery debate. Cambridge, Mass.: The MIT Press. Kosslyn, S. M. (2005). Mental images and the brain. Cognitive Neuropsychology, 22(3/4), 333-347. Kosslyn, S. M., Ball, T. M., & Reiser, B. J. (1978). Visual images preserve metric spatial information: Evidence from studies of image scanning. Journal of Experimental Psychology: Human Perception and Performance, 4(1), 47-60. Kosslyn, S. M., & Pomerantz, J. R. (1977). Imagery, propositions, and the form of internal representations. Cognitive Psychology, 9, 52-76. Kosslyn, S. M., Thompson, W. L., Kim, I. J., & Alpert, N. M. (1995). Topographical representations of mental images in primary visual cortex. Nature, 378, 496-498.

90

Kozhevnikov, M., Kosslyn, S. M., & Shephard, J. (2005). Spatial versus object visualizers: A new characterization of visual cognitive style. Memory & Cognition, 33(4), 710-726. Lane, D. M., & Robertson, L. (1979). The generality of the levels of processing hypothesis: An application to memory for chess positions. Memory & Cognition, 7(4), 253-256. Lewandowsky, S. (1999). Redintegration and response suppression in serial recall: A dynamic network model. International Journal of Psychology, 34(5/6), 434-446. Lockhart, R. S., & Craik, F. I. M. (1990). Levels of processing: A retrospective commentary on a framework for memory research. Canadian Journal of Psychology, 44(1), 87-112. Logie, R. H. (1995). Visuo-spatial working memory. Hove, UK: Lawrence Erlbaum Associates. Logie, R. H., Cornoldi, C., Brandimonte, M. A., Kaufmann, G., & Reisberg, D. (1996). Ducks, rabbits, and hedgehogs: Resolution, impasse, or fostered debate? In C. Cornoldi, R. H. Logie, M. A. Brandimonte, G. Kaufmann & D. Reisberg (Eds.), Stretching the imagination. Representation and transformation in mental imagery. (pp. 173-193). New York: Oxford University Press. Logie, R. H., & Edworthy, J. (1986). Shared mechanisms in the processing of verbal and musical material. In D. G. Russell, D. F. Marks & J. T. E. Richardson (Eds.), Imagery 2. (Vol. 2, pp. 33-37). New Zealand: Human Performance Associates. Logie, R. H., Engelkamp, J., Dehn, D., & Rudkin, S. (2001). Actions, mental actions, and working memory. In M. Denis, R. H. Logie, C. Cornoldi, M. De Vega & J. Engelkamp (Eds.), Imagery, Language and Visuo-spatial thinking (pp. 161-183). Hove: Psychology Press. Logie, R. H., Zucco, G., & Baddeley, A. D. (1990). Interference with visual short-term memory. Acta Psychologica, 75, 55-74. Maguire, E. A., Frackowiak, R. S. J., & Frith, C. D. (1997). Recalling routes around London: Activation of the right hippocampus in taxi drivers. Journal of Neuroscience, 17(18), 7103-7110. Marks, D. (1983). Mental imagery and consciousness: A theoretical review. In A. A. Sheikh (Ed.), Imagery: Current theory, research and application. New York: John Wiley. Marschark, M., Richman, C. L., Yuille, J. C., & Hunt, R. R. (1987). The role of imagery in memory: On shared and distinctive information. Psychological Bulletin, 102(1), 28-41. Marschark, M., & Surian, L. (1989). Why does imagery improve memory? European Journal of Cognitive Psychology, 1(3), 251-263. Marschark, M., & Surian, L. (1992). Concreteness effects in free recall: The roles of imaginal and relational processing. Memory & Cognition, 20(6), 612-620. Mast, F. W., & Kosslyn, S. M. (2002). Visual mental images can be ambiguous: insights from individual differences in spatial transformation abilities. Cognition, 86, 57-70. Meinz, E. J., & Salthouse, T. A. (1998). The effects of age and experience on memory for visually presented music. Journal of Gerontology: Psychological Sciences, 53B(1), P60P69. Mikumo, M. (1994). Motor encoding strategy for pitches of melodies. Music Perception, 12(2), 175-197. Miller, G. A. (1956). The magical number seven plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81-97. Nairne, J. S. (1990). A feature model of immediate memory. Memory & Cognition, 18(3), 251-269. Nairne, J. S. (2002). Remembering over the short-term: The case against the standard model. Annual Review of Psychology, 53, 53-81. Neisser, U. (1972). Changing conceptions of imagery. In P. W. Sheehan (Ed.), The functions and nature of imagery (pp. 233-251). New York: Academic Press. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, N.J.: Prentice Hall. Paivio, A. (1971). Imagery and verbal processes. New York: Holt, Rinehart and Winston, Inc.

91

Paivio, A. (1986). Mental representations, a dual coding approach. New York: Oxford University Press. Paivio, A. (1991). Images in mind: The evolution of a theory. New York: Harvester Wheat Sheaf. Palmer, S. E. (1977). Hierarchial Structure in Perceptual Representation. Cognitive Psychology, 9, 441-474. Pearson, D., De Beni, R., & Cornoldi, C. (2001). The generation, maintenance, and transformation of visuo-spatial mental images. In M. Denis, R. H. Logie, C. Cornoldi, M. De Vega & J. Engelkamp (Eds.), Imagery, Language and Visuo-spatial thinking (pp. 1-27). Hove: Psychology Press. Pechmann, T., & Mohr, G. (1992). Interference in memory for tonal pitch: Implications for a working-memory model. Memory & Cognition, 20(3), 314-320. Peretz, I., & Kolinsky, R. (1993). Boundaries of separability between melody and rhythm in music discrimination: a neuropsychological perspective. Quarterly Journal of Experimental Psychology, 46A(2), 301-325. Perrig, W. J., & Hofer, D. (1989). Sensory and conceptual representations in memory: Motor images that cannot be imagined. Psychological Research, 51, 201-207. Pruch, P., Giraudo, M.-D., & Grling, T. (1989). Distance cognition by taxi drivers and the general public. Journal of Environmental Psychology, 9, 233-239. Povel, D.-J., & Jansen, E. (2001). Perceptual mechanisms in music processing. Music Perception, 19(2), 169-198. Pylyshyn, Z. W. (1973). What the mind's eye tells the mind's brain: A critique of mental imagery. Psychological Bulletin, 80(1), 1-24. Pylyshyn, Z. W. (1981). The imagery debate: Analog media versus tacit knowledge. In N. Block (Ed.), Imagery. Cabridge, MA: The MIT Press. Pylyshyn, Z. W. (2002). Mental imagery: In search of a theory. Bahavioral and Brain Sciences, 25, 157-238. Reed, C. L. (2002). Chronometric comparisons of imagery to action: Visualizing versus physically performing springboard dives. Memory & Cognition, 30(8), 1169-1178. Reingold, E. M., Charness, N., Pomplun, M., & Stampe, D. M. (2001). Visual span in expert chess players: evidence from eye movements. Psychological Science, 12(1), 48-55. Reisberg, D. (1996). The nonambiguity of mental images. In C. Cornoldi, R. H. Logie, M. A. Brandimonte, G. Kaufmann & D. Reisberg (Eds.), Stretching the imagination. Representation and transformation in mental imagery. (pp. 119-171). New York: Oxford University Press. Reisberg, D., Smith, J. D., Baxter, D. A., & Sonenshine, M. (1989). "Enacted" auditory images are ambiguous; "pure" auditory images are not. The Quarterly Journal of Experimental Psychology, 41A(3), 619-641. Reisberg, D., Wilson, M., & Smith, J. D. (1991). Auditory imagery and inner speech. In R. Logie, H. & M. Denis (Eds.), Mental images in human cognition (pp. 59-81): Elsevier Science Publishers B.V. Reitman, J. S. (1976). Skilled perception in go: deducing memory structures from interresponse times. Cognitive Psychology, 8, 336-356. Richardson, J. T. E. (1980). Mental imagery and human memory. London: Macmillan. Richman, H. B., Staszewski, J. J., & Simon, H. A. (1995). Simulation of expert memory with EPAM IV. Psychological Review, 102, 305-330. Robbins, T. W., Anderson, E. J., Barker, D. R., Bradley, A. C., Fearnyhough, C., Henson, R., et al. (1996). Working memory in chess. Memory & Cognition, 24(1), 83-93. Roberts, L. A. (1986). Modality and suffix effects in memory for melodic and harmonic musical materials. Cognitive Psychology, 18, 123-157. Rouw, R., Kosslyn, S. M., & Hamel, R. (1997). Detecting high-level and low-level properties in visual images and visual percepts. Cognition, 63, 209-226.

92

Ruchkin, D. S., Grafman, J., Cameron, K., & Berndt, R. S. (2003). Working memory retention systems: A state of activated long-term memory. Behavioral and Brain Sciences, 26, 709-777. Saariluoma, P. (1984). Coding problem spaces in chess: A psychological study (Vol. 23). Helsinki: Finnish Society of Scineces and Letters. Saariluoma, P. (1989). Chess players' recall of auditorily presented chess positions. European Journal of Cognitive Psychology, 1, 309-320. Saariluoma, P. (1991). Aspects of skilled imagery in blindfold chess. Acta Psychologica, 77, 65-89. Saariluoma, P. (1992a). Do visual images have gestalt properties? The Quarterly Journal of Experimental Psychology, 45A(3), 399-420. Saariluoma, P. (1992b). Error in chess: apperception restructuring view. Psychological Research, 54, 17-26. Saariluoma, P. (1992c). Visuo-spatial and articulatory interference in chess players' information intake. Applied Cognitive Psychology, 6, 77-89. Saariluoma, P. (1995). Chess players' thinking. London: Routledge. Saariluoma, P. (1997). Foundational analysis. Presuppositions in experimental psychology. London: Routledge. Saariluoma, P., Karlsson, H., Lyytinen, H., Ters, M., & Geisler, F. (2004). Visuospatial representations udes by chess experts: A preliminary study. European Journal of Cognitive Psychology, 16(5), 753-766. Salam, P., & Baddeley, A. D. (1989). Effects of background music on phonological shortterm memory. Quarterely Journal of Experimental Psychology, 41A, 107-122. Salthouse, T. A., Babcock, R. L., Skovronek, E., Mitchell, D. R. D., & Palmon, R. (1990). Age and experience effects in spatial visualization. Developmental Psychology, 26(1), 128136. Schweickert, R. (1993). A multinomial processing tree model for degradation and redintegration in immediate recall. Memory & Cognition, 21(2), 168-175. Schweickert, R., Chen, S., & Poirier, M. (1999). Redintegration and the useful lifetime of the verbal memory representation. International Journal of Psychology, 34(5/6), 447-453. Schrmann, M., Raij, T., Fujiki, N., & Hari, R. (2002). Mind's ear in a musician: Where and when in the brain. NeuroImage, 16, 434-440. Shah, P., & Miyake, A. (1999). Models of working memory. In A. Miyake & P. Shah (Eds.), Models of working memory: Mechanisms of active maintenance and executive control (pp. 1-27). New York: Cambridge University Press. Simon, H. A., & Gilmartin, K. (1973). A simulation of memory for chess positions. Cognitive Psychology, 5, 29-46. Simon, H. A., & Gobet, F. (2000). Expertise effects in memory recall: Comments on Vicente and Wang 1998. Psychological Review, 107(3), 593-600. Sloboda, J. A. (1976). Visual perception of musical notation: Registering pitch symbols in memory. Quarterly Journal of Experimental Psychology, 28, 1-16. Sloboda, J. A. (1984). Experimental studies of music reading: A review. Music Perception, 2, 222-236. Sloboda, J. A. (1985). The musical mind: The cognitive psychology of music. London: Oxford University Press. Stigler, J., W. (1984). "Mental abacus": The effect of abacus training on Chinese children's mental calculation. Cognitive Psychology, 16, 145-176. Stuart, G., & Hulme, C. (2000). The effects of word co-occurence on short-term memory: Associative links in long-term memory affect short-term memory performance. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(3), 796-802. Thorn, A. S. C., Gathercole, S. E., & Frankish, C. R. (2005). Redintegration and the benefits of long-term knowledge in verbal short-term memory: An evaluation of Schweickert's (1993) multinomial processing tree model. Cognitive Psychology, 50, 133-158.

93

Tillmann, B., Bharucha, J. J., & Bigand, E. (2000). Implicit learning of tonality: A selforganizing approach. Psychological Review, 107(4), 885-913. Waters, A. J., Gobet, F., & Leyden, G. (2002). Visuospatial abilities of chess players. British Journal of Psychology, 93, 557-565. Waters, A. J., Underwood, G., & Findlay, J. M. (1997). Studying expertise in music reading: Use of a pattern-matching paradigm. Perception & Psychophysics, 59(4), 477-488. Verhaegen, P., & Salthouse, T. A. (1997). Meta-analyses of age cognition relations in adulthood: Estimates of linear and nonlinear age effect and structural models. Psychological Bulletin, 122(3), 231-249. Vicente, K. J. (2000). Revisiting the constraint attunement hypothesis: A reply to Ericsson, Patel, & Kintsch (2000) and Simon & Gobet (2000). Psychological Review, 107(3), 601608. Vicente, K. J., & Wang, J. H. (1998). An ecological theory of expertise effects in memory recall. Psychological Review, 105(1), 33-57. Williamon, A., & Egner, T. (2004). Memory structures for encoding and retrieving a piece of music: an ERP investigation. Cognitive Brain Research, 22, 36-44. Williamon, A., & Valentine, E. (2002). The role of retrieval structures in memorizing music. Cognitive Psychology, 44(1), 1-32. Zatorre, R. J., Halpern, A. R., Perry, D. W., Meyer, E., & Evans, A. C. (1996). Hearing in the mind's ear: A PET investigation of musical imagery and perception. Journal of Cognitive Neuroscience, 8(1), 29-46. Zhang, G., & Simon, H. A. (1985). STM capacity for Chinese words and idioms: Chunking and acoustical loop hypoteses. memory & Cognition, 13(3), 193-201.