LANGUAGE AND COGNITIVE PROCESSES, 1996, 11 (6), 569576

Cross-Modal Semantic Priming

Patrizia Tabossi
University of Trieste, Trieste, Italy
The cross-modal semantic priming paradigm is described, including its underlying rationale and the different tasks with which it is combined. The major characteristics of the paradigmthe type of stimuli used, the dependent and independent variables typically manipulatedare then introduced. Also, its main advantages and drawbacks are discussed. Finally, the most important areas of application are considered and some important ndings which have been obtained with it are brie y mentioned.

Issues Addressed
Access to ambiguous words, spoken word processing, speech segmentation and sentence processing.

First Uses
Originally introduced by Warren (1972). In its most common form, it was proposed and tested by Swinney, Onifer, Prather and Hirshkowitz (1979).

Description
Subjects listen to a spoken stimulus (prime). At various times relative to prime presentation, a visual target which may be semantically related/ associated or unrelated to the primeis presented to them. Primes are usually embedded in sentential contexts. Typically, subjects perform a lexical decision task on the visual target. Also, to make sure that subjects pay attention to the acoustic materials, they are given a recognition test either at various points during the experimental session or at the end.
Requests for reprints should be addressed to P. Tabossi, Dipartimento di Psicologia, Via dellUniversita 7, 34123 Trieste, Italy. E-mail: tabossi6 psibo.unibo.it This research was supported in part by CNR Grant Nos 94.04060/ 08 and 95.01841H5. I would like to thank Anne Cutler and the reviewers for their helpful comments.
q 1996 Psychology Press, an imprint of Erlbaum (UK) Taylor & Francis Ltd

570

TABOSSI

Stimuli
Primes are spoken ambiguous words (Swinney, 1979), unambiguous words, ambiguous or unambiguous word fragments (Zwitserlood, 1989), fragments of segmentally ambiguous speech (Shillcock, 1990; Gow & Gordon, 1995; Tabossi, Burani, & Scott, 1995) or structurally relevant positions in sentences, including verbs, pronouns and gaps (Shapiro, Zurif, & Grimshaw, 1987, 1988; MacDonald & MacWhinney, 1990). Targets are visual words or pseudoword s.

Dependent Variables
1. Lexical decision RTs and errors. 2. Naming RTs (e.g. Seidenberg, Tanenhaus, Leiman, & Bienkowski, 1982). 3. Other, e.g. subjects name the colour of a target word (Warren, 1972; Conrad, 1974; Oden & Spira, 1983; Whitney, McKay, Kellas, & Emerson, 1985). This is known as the Stroop task.

Independen t Variables
Ambiguous words
1. Types of lexical ambiguity: category (e.g. noun/verb: Seidenberg et al., 1982); homophony (Cutler, 1986); balanced or unbalanced meanings (dominance) (Onifer & Swinney, 1981). 2. Types of context: neutral or biasing sentential context (Swinney, 1979); types of sentential constraints (Tabossi, 1988a). 3. Timing of target presentation relative to prime presentation: at the offset of the ambiguous word (Swinney, 1979), prior to offset (Tabossi & Zardon, 1993), after offset (Swinney, 1979; Simpson, 1981). 4. Relation between prime and target: association, semantic features (Tabossi, Colombo, & Job, 1987; Tabossi, 1988a; Swinney, 1991).

Unambiguous words and word fragments

1. Words and word fragments: acoustic/phonetic characteristics of the prime (Andruski, Blumstein, & Burton, 1994; Connine, 1994; Connine, Blasko, & Wang, 1994); nature and amount of acoustic/phonetic similarity with other words in the language (Marslen-W ilson & Zwitserlood, 1989; Connine, Blasko, & Titone, 1993); number of candidates similar to the prime (Andruski et al., 1994; Connine, 1994); identi cation point (Zwitserlood, 1989). 2. Types of context: single word (Marslen-Wilson & Zwitserlood, 1989); biasing and neutral sentential contexts (Zwitserlood, 1989; Connine et al.,

CROSS-MODAL SEMANTIC PRIM ING

571

1994); segmentally ambiguous contexts and embedding (Shillcock, 1990; Gow & Gordon, 1995; Tabossi et al., 1995). 3. Timing: moment (during or after) with respect to spoken stimulus presentation that visual target is presented (Zwitserlood, 1989; Andruski et al., 1994; Connine, 1994). 4. Relation between prime and target: associations vs semantic features (Whitney et al., 1985; Tabossi, 1988b); acoustic/phonetic similarity (Connine et al., 1993); rhyming (Marslen-Wilson & Zwitserlood, 1989).

Sentence processing
1. Pronoun ambiguity (MacDonald & MacWhinney, 1990). 2. Loci of trace activation (Nicol & Swinney, 1989). 3. Verb complexity and lexical effects (Shapiro et al., 1987, 1988; Boland, 1991).

Analysis Issues
1. The existence (and amount) of facilitatory priming of the visual decision is assumed to provide information about the processing of the auditory prime. Furthermore, prior sentential context is assumed to in uence the prime, and only through this in uence to affect performance on the visual target (Swinney, 1979; Tabossi, 1988a, 1988b). 2. The facilitation that the prime produces on the target processing is measured in two ways: Either by comparing performance on the same target preceded by a related prime in one condition and by an unrelated prime in the control condition (Marslen-Wilson & Zwitserlood, 1989; Whitney et al., 1985; Tabossi, 1988b), or by comparing performance on different targets, one related and one unrelated to the same prime (Swinney, 1979; Onifer and Swinney, 1981). In the latter case, performance on the targets outside the experimental conditions must be comparable (similar length, frequency, etc.).

Effects Found with Paradigm

Ambiguous words
1. Lack of syntactic context effects on access Shown by: Seidenberg et al. (1982). 2. Effects of dominance Shown by: Tabossi et al. (1987; but see Onifer & Swinney, 1981). 3. Fast post-access selective effects of sentential context Shown by: Swinney (1979).

572

TABOSSI

4. Some context effects on initial access Shown by: Simpson (1981); Tabossi (1988a); Tabossi and Zardon (1993).

Unambiguous words/word fragments

1. Multiple early activation of competitors Shown by: Zwitserlood (1989). 2. Partially matching information can activate lexical candidates Shown by: Andruski et al. (1994); Connine et al. (1994). 3. Various context effects Shown by: Zwitserlood (1989). 4. Relevant perceptual characteristics of the initial fragment of the to-beidenti ed word Shown by: Connine et al. (1993); Marslen-Wilson and Zwitserlood (1989). 5. Context-dependent and context-independent activation of aspects of word meanings Shown by: Whitney et al. (1985); Greenspan (1986); Tabossi (1988b); Williams (1988). 6. Effects of segmental ambiguity within and across word-boundaries Shown by: Shillcock (1990); Gow and Gordon (1995); Tabossi et al. (1995).

Sentence processing
1. Assignment of pronoun reference can be slow Shown by: MacDonald and MacWhinney (1990). 2. Selective activation of structurally appropriate antecedents Shown by: Nicol (1988); Nicol and Swinney (1989). 3. Immediate activation of all argument structure for verbs during sentence processing Shown by: Shapiro et al. (1987, 1988). 4. Early effects of lexical information on sentence processing Shown by: Boland (1991).

Design Issues
1. Single word (e.g. Marslen-Wilson & Zwitserlood, 1989; Andruski et al., 1994) versus sentential context (e.g. Swinney, 1979; Whitney et al., 1985). 2. Subjects task: colour naming (Warren, 1972; Whitney et al., 1985), lexical decision (Swinney et al., 1979), etc.

CROSS-MODAL SEMANTIC PRIM ING

573

3. Presentation of the target relative to the prime: at prime offset (Swinney, 1979; Zwitserlood, 1989); after prime ending at different inter-stimulus intervals (ISI) (Swinney, 1979; Simpson, 1981; Whitney et al., 1985; Andruski et al., 1994; Connine, 1994); before word ending (Tabossi & Zardon, 1993). 4. Full word vs word fragment: when presented before the offset of the acoustic prime, the prime presentation is often interrupted at the occurrence of the target. In this case, the prime is actually a fragment whose length may correspond to different points, such as isolation, uniqueness or recognition point (Zwitserlood, 1989).

Validity
1. The paradigm is reasonably reliable. 2. Its rationale has resisted careful scrutiny. 3. It re ects local phenom ena rather than global sentence effects.

Advantages
1. It relies on a robust phenomenon (semantic priming). 2. It taps semantic activation produced by spoken stimuli on-line. 3. There is little interference with the on-going process of comprehension by the task(s). 4. It is very accurate time-wise.

Potential Artifacts
1. There may be backward priming effects, although these have not unambiguously been demonstrated with auditory primes or sentential contexts (Burgess, Seidenberg, & Tanenhaus, 1986; Glucksberg, Kreuz, & Rho, 1986; Peterson & Simpson, 1989). 2. With sentential contexts, it is necessary to control the semantic relatedness between target and individual words of the preceding context, even though there is no clear evidence that such relatedness affects task performance (Swinney, 1979; Tabossi, 1988b).

Problems
1. Possible contaminating effect of backward priming effects could be reduced by using naming rather than the lexical decision task. However, naming is problematic because of its possible interference with speech perception.

574

TABOSSI

2. Potential differences between results obtained on targets presented at the offset of word-fragment primes or at the same point of a complete word are so far unclear. 3. In many cases, the decision of where to place a timing pulse in the continuous speech signal is somewhat arbitrary. 4. Selection and testing of appropriate materials is dif cult and timeconsuming. Also, constraintsboth theoretical and practical reduce the number of well-controlled experim ental items. 5. Variability between and within subjects is often high, more so than between and within items.

Uses with Other Populations

Cross-modal semantic priming has been used with asphasic patients. The following have been studied, among others: access to ambiguous words (Prather, Shapiro, Zurif, & Swinney, 1991), gap- lling ability (Zurif et al., 1993; Zurif & Swinney, 1994) and effects of verb argument structure (Shapiro, Gordon, Hack, & Killackey, 1993) in Brocas and Wernickes aphasics.

References
Andruski, J.E., Blumstein, S.E., & Burton, M. (1994). The effect of subphonetic differences on lexical access. Cognition , 52, 163187. Boland, J.E. (1991). The use of lexical knowledge in sentence processing. Doctoral dissertation, University of Rochester, Rochester, NY. Burgess, C., Seidenberg, M., & Tanenhaus, M. (1986). Context and lexical access: Implications of nonword interference for lexical ambiguity resolution. Journal of Experimental Psychology: Learning, Memory and Cognition, 15 , 620632. Connine, C. (1994). Vertical and horizontal similarity in spoken-word recognition. In C. Clifton, L. Frazier, & K. Rayner (Eds), Perspectives on sentence processing, pp. 107120. Hillsdale, NJ: Lawrence Erlbaum Associates Inc. Connine, C., Blasko, D., & Titone, D. (1993). Do the beginnings of spoken words have a special status in auditory word recognition? Journal of Memory and Language, 32, 193210. Connine, C., Blasko, D., & Wang, J. (1994). Vertical similarity in spoken word recognition: Multiple lexical activation, individual differences, and the role of sentence context. Perception and Psychophysics, 56, 624636. Conrad, C. (1974). Context effects in sentence comprehension: A study of the subjective lexicon. Memory and Cognition, 2 , 130138. Cutler, A. (1986). Forbear is not a homophone: Lexical prosody does not constrain lexical access. Language and Speech, 29, 201220. Glucksberg, S., Kreuz, R.J., & Rho, S.H. (1986). Context can constrain lexical access: Implications for models of language comprehension. Journal of Experimental Psychology: Learning, Memory and Cognition, 12 , 323335. Gow, D.W . Jr, & Gordon, P.C. (1995). Lexical and prelexical in uences on word segmentation : Evidence from priming. Journal of Experimental Psychology: Human Perception and Performance, 21, 344359. Greenspan, S. (1986). Semantic exibility and referential speci city of concrete nouns. Journal of Memory and Language, 25, 539557.

CROSS-MODAL SEMANTIC PRIM ING

575

MacDonald, M.C., & MacWhinney, B. (1990). Measuring inhibition and facilitation from pronouns. Journal of Memory and Language, 29, 469492. Marslen-W ilson, W., & Zwitserlood, P. (1989). Accessing spoken words: On the importance of word onset. Journal of Experimental Psychology: Human Perception and Performance, 15, 576585. Nicol, J. (1988). Coreference processing during sentence comprehension. Doctoral dissertation, MIT, Cambridge, MA. Nicol, J., & Swinney, D. (1989). The role of structure in coreference assignment during sentence comprehension. Journal of Psycholinguistic Research, 18 , 520. Oden, G.C., & Spira, J.L. (1983). In uence of context on the activation and selection of ambiguous word senses. Quarterly Journal of Experimental Psychology, 35, 5164. Onifer, W., & Swinney, D. (1981). Accessing lexical ambiguity during sentence comprehension: Effects of frequency of meaning and contextual bias. Memory and Cognition, 9, 225236. Peterson, R., & Simpson, G. (1989). Effects of backward priming on word recognition in single word and sentence contexts. Journal of Experimental Psychology: Learning, Memory and Cognition, 15, 1020 1032. Prather, P., Shapiro, L., Zurif, E., & Swinney, D. (1991). Real-time examination of lexical processing in aphasics. Journal of Psycholinguistic Research, 23, 271281. Seidenberg, M., Tanenhaus, M., Leiman, J., & Bienkowski, M. (1982). Automatic access of the meanings of ambiguous words in context: Some limitations of the knowledge-based processing. Cognitive Psychology, 14, 489537. Shapiro, L.P., Zurif, E., & Grimshaw, J. (1987). Sentence processing and the mental representation of verbs. Cognition, 27 , 219246. Shapiro, L.P., Zurif, E., & Grimshaw, J. (1988). Verb processing during sentence comprehension: Contextual impenetrabilit y. Journal of Psycholinguistic Research, 18 , 223243. Shapiro, L., Gordon, B., Hack, N., & Killackey, J. (1993). Argument structure processing in complex sentences in Brocas and Wernickes aphasia. Brain and Language, 45 , 423447. Shillcock, R. (1990). Lexical hypotheses in continuous speech. In G.T.M . Altmann (Ed.), Cognitive models of speech processing, pp. 2449. Cambridge, MA: MIT Press. Simpson, G. (1981). Meaning dominance and semantic context in the processing of lexical ambiguity. Journal of Verbal Learning and Verbal Behavior, 20 , 120136. Swinney, D. (1979). Lexical access during sentence comprehension: (Re)consideration of context effects. Journal of Verbal Learning and Verbal Behavior, 18, 645659. Swinney, D. (1991). The resolution of indeterminancy during language comprehension: Perspectives on modularity in lexical, structural and pragmatic processing. In G.B. Simpson (Ed.), Understanding word and sentence, pp. 367385. Amsterdam: North-Holland. Swinney, D., Onifer, W., Prather, P., & Hirshkowitz, M. (1979). Semantic facilitation across modalities in the processing of individual words and sentences. Memory and Cognition, 7, 159165. Tabossi, P. (1988a). Accessing lexical ambiguity in different types of sentential contexts. Journal of Memory and Language, 27, 324340. Tabossi, P. (1988b). Effects of context on the immediate interpretation of unambiguous nouns. Journal of Experimental Psychology: Learning, Memory and Cognition, 14, 153162. Tabossi, P., & Zardon, F. (1993). Processing ambiguous words in context. Journal of Memory and Language, 32, 359372. Tabossi, P., Colombo, L., & Job, R. (1987). Accessing lexical ambiguity: Effects of context and dominance. Psychological Research, 49, 161167. Tabossi, P., Burani, C., & Scott, D. (1995). Word identi cation in uent speech. Journal of Memory and Language, 34, 440467. Warren, R.E. (1972). Stimulus encoding and memory. Journal of Experimental Psychology, 94, 90100.

576

TABOSSI

Whitney, P., McKay, T., Kellas, G., & Emerson, W.A. Jr (1985). Semantic activation of noun concepts in context. Journal of Experimental Psychology: Learning, Memory and Cognition, 11, 126135. Williams, J. (1988). Semantic activation during sentence comprehension. Language and Cognitive Processes, 3, 165206. Zurif, E., & Swinney, D. (1994). The neuropsychology of language. In M.A. Gernsbacher (Ed.), Handbook of psycholinguistics, pp. 1055 1074. New York: Academic Press. Zurif, E., Swinney, D., Prather, P., Solomon, J., & Bushell, C. (1993). An on-line analysis of syntactic processing in Brocas and Wernickes aphasia. Brain and Language, 45 , 448464. Zwitserlood, P. (1989). The locus of the effects of sentential-semantic context in spokenword processing. Cognition, 32 , 2564.