Assessing The Role of Visual and Auditory Cues in Multisensory Perception of Flavor

the Role of
37 Assessing
Visual and Auditory
Cues in Multisensory
Perception of Flavor
Massimiliano Zampini and Charles Spence
CONTENTS
37.1 Introduction........................................................................................................................... 739
37.2 Multisensory Interactions between Visual and Flavor Perception........................................ 740
37.2.1 Role of Color Cues on Multisensory Flavor Perception............................................ 740
37.2.2 Color-Flavor Interactions: Possible Role of Taster Status.......................................... 743
37.2.3 ColorFlavor Interactions: Possible Role of Learned Associations between
Colors and Flavors..................................................................................................... 745
37.2.4 ColorFlavor Interactions: Neural Correlates........................................................... 747
37.2.5 Interim Summary...................................................................................................... 748
37.3 Role of Auditory Cues in the Multisensory Experience of Foodstuffs................................. 749
37.3.1 Effect of Sound Manipulation on the Perception of Crisps....................................... 749
37.3.2 Effect of Auditory Cues on the Perception of Sparkling Water................................ 751
37.4 Conclusions............................................................................................................................ 752
References....................................................................................................................................... 753
37.1 INTRODUCTION
Our perception of the objects and events that fill the world in which we live depends on the integration
of the sensory inputs that simultaneously reach our various sensory systems (e.g., vision, audition,
touch, taste, and smell). Perhaps the best-known examples of genuinely multisensory experiences
come from our perception and evaluation of food and drink. The average person would say that the
flavor of food derives primarily from its taste in the mouth. They are often surprised to discover that
there is a strong nasal role in the perception of flavor. In fact, it has been argued that the majority
of the flavor of food actually comes from its smell (e.g., Cain 1977; Murphy and Cain 1980; Rozin
1982).* Our perception of food and drink, however, is not simply a matter of combining gustatory
* For example, coffee and tea are indistinguishable (with both having a bitter taste) if drunk while holding ones nose
pinched shut. Whereas the taste of a lemon only actually consists of sour, sweet, and bitter components, most of the flavor
we normally associate with the taste of a lemon actually comes from the terpene aroma, one of the constituent chemicals
that stimulate the olfactory mucosa via the nasopharynx (i.e., retronasal olfaction). Odor molecules may reach the receptors in the olfactory epithelium (i.e., the area located in the rear of the nasal cavity) traveling inward from the anterior
nares or through the posterior nares of the nasopharynx. Most typically, orthonasal olfaction occurs during respiratory
inhalation or sniffing, whereas retronasal olfaction occurs during respiratory exhalation or after swallowing. People
usually report experiencing odors as originating from the external world when perceived orthonasally, and as coming
from the mouth when perceived retronasally (Rozin 1982). Importantly, the latest cognitive neuroscience evidence has
highlighted the fact that somewhat different neural structures are used to process these two kinds of olfactory information (Small et al. 2005, 2008; see also Koza et al. 2005).
739
Downloaded by [University of Cincinnati] at 09:36 20 May 2016
740
The Neural Bases of Multisensory Processes
and olfactory food cues (although this is undoubtedly very important; Dalton et al. 2000). For
instance, our evaluation of the pleasantness of a particular foodstuff can be influenced not only by
what it looks, smells, and tastes like, but also what it sounds like in the mouth (think, for example, of
the auditory sensations associated with biting into a potato chip or a stick of celery; see Spence and
Zampini 2006, for a review). The feel of a foodstuff (i.e., its oralsomatosensory attributes) is also
very important; the texture, temperature, viscosity, and even the painful sensations we experience
when eating hot foods (e.g., chilli peppers) all contribute to our overall multisensory experience of
foodstuffs (e.g., Bourne 1982; Lawless et al. 1985; Tyle 1993). Flavor perception is also influenced
by the interactions taking place between oral texture and both olfactory and gustatory cues (see also
Bult et al. 2007; Christensen 1980a, 1980b; Hollowood et al. 2002). Given the multisensory nature
of our perception of food, it should come as little surprise that many studies have been conducted in
order to try and understand the relative contribution of each sense to our overall evaluation of food
(e.g., see Delwiche 2004; Spence 2002; Stevenson 2009; Stillman 2002). In this chapter, we review
the contribution of visual and auditory cues to the multisensory perception of food. Moreover, any
possible influence of visual and auditory aspects of foods and drinks might take place at different
stages of the food experience. Visual cues are perceived when foodstuffs are outside of the mouth.
Auditory cues are typically primarily perceived when we are actually consuming food.
37.2 M
ULTISENSORY INTERACTIONS BETWEEN
VISUAL AND FLAVOR PERCEPTION
37.2.1 Role of Color Cues on Multisensory Flavor Perception
Over the past 80 years or so, many researchers have been interested in the role of visual information
in the perception of foodstuffs (Moir 1936). It seems that the visual appearance of food and drink can
have a profound impact on our perception and evaluation of flavor. The role of color cues on peoples
flavor perception has been investigated in many different studies, although the majority of the research
has been published in food science journals rather than psychology or neuroscience (for reviews, see
Clydesdale 1993; Delwiche 2004; Spence et al. 2010; Stevenson 2009). The majority of these studies
have shown that peoples perception of a variety of different foods and drinks can be dramatically
modified by changing the color of food or drink items (e.g., DuBose et al. 1980; Duncker 1939; Garber
et al. 2000; Johnson and Clydesdale 1982; Morrot et al. 2001; Philipsen et al. 1995; Roth et al. 1988;
Stillman 1993; Wheatley 1973; Zampini et al. 2007; Zellner and Durlach 2003).
One of the most dramatic early empirical demonstrations of the strong link between color
and the pleasure we derive from food (and/or our appetitive responses to food) was reported by
Wheatley (1973). He described a situation in which a group of people ate a meal of steak, French
fries, and peas under color-masking lighting conditions. Halfway through the meal, normal lighting
was restored revealing that the steak was colored blue, the French fries had been colored green, and
the peas were red. According to Wheatleys description, the mere sight of the food was sufficient to
induce nausea in many of his dinner guests. Such results, although anecdotal, do at least hint at the
powerful influence that visual cues can have over our appetitive responses.
Color has also been shown to exert a significant effect on our ability to recognize specific foodstuffs. For example, in one oft-cited study, DuBose et al. (1980) presented participants with drinks
incorporating a variety of different colorflavor combinations (the flavored solutions were colored
either appropriately, inappropriately, or else were presented as colorless solutions). DuBose et al.
found that participants identification of the flavors of many of the drinks was significantly influenced by their color. In particular, the participants were less accurate in identifying the flavor of
fruit-flavored beverages when they were unaware of the appropriate color. For instance, 40% of
the participants reported that a cherry-flavored beverage actually tasted of orange when it had
been inappropriately colored orange (compared to 0% orange-flavor responses when the drink was
appropriately colored red; a similar effect was reported for the lime-flavored beverage). Many other
Assessing the Role of Visual and Auditory Cues in Multisensory Perception of Flavor
741
researchers have reported a similar visual modulation of participants odor discrimination/identification responses (e.g., Blackwell 1995; Davis 1981; Koza et al. 2005; Morrot et al. 2001; Stevenson
and Oaten 2008; Zellner et al. 1991; Zellner and Kautz 1990; Zellner and Whitten 1999).
Although the potential influence of color cues on peoples flavor identification responses is by
now well documented, the evidence regarding the impact of changes in color intensity on perceived
flavor intensity is rather less clear. For example, ambiguous results have been reported in studies in
which the participants had to rate the intensity of the flavor of solutions that varied in the intensity
of the color that had been added to the solutions (e.g., DuBose et al. 1980; Johnson and Clydesdale
1982; Johnson et al. 1983; see Clydesdale 1993, for a review). For example, DuBose et al. found
that overall flavor intensity was affected by color intensity, with more intense coloring resulting in
stronger flavor evaluation responses by participants for the orange-flavored, but not for the cherryflavored beverages, tested in their study. However, in other studies, the concentration of coloring
in the solutions did not influence participants ratings, regardless of whether the solutions were
appropriately or inappropriately colored (e.g., Alley and Alley 1998; Frank et al. 1989; Zampini et
al. 2007).
Researchers have also investigated the effect of varying the intensity of the color on the perceived intensity of tastes and odors separately. For instance, the addition of a red coloring to cherryand strawberry-flavored sucrose solutions has been found to increase the perceived sweetness of
these solutions in certain studies (Johnson and Clydesdale 1982; Johnson et al. 1983). Maga (1974)
hypothesized that the influence of colors on sweetness perception in humans might be particularly
strong for colors that are typically associated with the natural ripening of fruits (e.g., yellow, red; see
also Lavin and Lawless 1998; Strugnell 1997). By contrast, researchers have reported that the addition of color has no effect on the perceived saltiness of foods such as soups (Gifford and Clydesdale
1986; Gifford et al. 1987; Maga 1974), perhaps because (in contrast to sweet foods) there are no
particular colors associated with the salt content of a food (i.e., salt is ubiquitous to many different
kinds, and hence colors, of food; see Maga 1974 and Lavin and Lawless 1998, on this point).
In one of the earliest studies to have been published in this area, Pangborn (1960) reported that
people reported green-colored pear nectar as being less sweet than colorless pear nectar. However,
Pangborn and Hansen (1963) failed to replicate these results. Although they found that green coloring had no effect on the perceived sweetness of pear nectar, its addition did give rise to an overall
increase in sensitivity to sweetness when color was added to the solutions. Similarly, for the pairing
of color with odor, Zellner and Kautz (1990) reported that solutions were rated as having a more
intense odor when color had been added to the solutions than when it was absent, regardless of the
appropriateness of the colorodor match. In fact, Zellner and Kautz noted that the participants in
their study simply refused to believe that colored and uncolored solutions of equal odor intensity
were actually equally strong.
The explanation for these contradictory results regarding the influence of variations in color
intensity on the perception of taste, odor, and flavor (i.e., odor + taste) intensity is far from obvious (see Shankar et al. 2010). For example, Chan and Kane-Martinelli (1997) reported that the
perceived flavor intensity for certain foods (such as chicken bouillon) was higher with the commercially available color sample than when the samples were given in a higher-intensity color (see
also Clydesdale 1993, on this point). Note also that if the discrepancy between the intensity of the
color and the intensity of the flavor is too great, participants may experience a disconfirmation of
expectation (or some form of dissonance between the visually and gustatorily determined flavor
intensities) and the color and taste cues may no longer be linked (e.g., Clydesdale 1993; cf. Ernst and
Banks 2002; Yeomans et al. 2008).
Another potentially important issue in terms of assessing interactions between color and flavor
is the role of peoples awareness of the congruency of the colorflavor pairings used (Zampini et al.
2007). In fact, in most of the research that has been published to date on the effects of color cues on
human flavor perception, the participants were not explicitly informed that the flavors of the solutions they were evaluating might not be paired with the appropriately colored solutions (e.g., see
742
DuBose et al. 1980; Johnson and Clydesdale 1982; Morrot et al. 2001; Oram et al. 1995; Philipsen
et al. 1995; Roth et al. 1988; Stillman 1993; Zellner and Durlach 2003). One might therefore argue
that the visual modulation of flavor perception reported in many of these previous studies simply
reflects a decisional bias introduced by the obvious variation in the color cues (cf. the literature on
the effectiveness of the color of medications on the placebo effect; e.g., de Craen et al. 1996; see also
Engen 1972), rather than a genuine perceptual effect (i.e., whereby the color cues actually modulate
the perception of flavor itself; although see also Garber et al. 2001, 2008, for an alternative perspective from the field of marketing). For example, if participants found it difficult to correctly identify the
flavor of the food or drink on the basis of gustatory and olfactory cues in flavor discrimination tasks,
then they may simply have based their responses on the more easily discriminable color cues instead.
Therefore, it might be argued that participants judgments in these previous studies may simply have
been influenced by decisional processes instead.
In their study, Zampini et al. (2007) tried to reduce any possible influence of response biases that
might emerge when studying colorflavor interactions by explicitly informing their participants that
the colorflavor link would often be misleading (i.e., that the solutions would frequently be presented
in an inappropriate color; cf. Bertelson and Aschersleben 1998). This experimental manipulation
was introduced in order to investigate whether the visual cues would still influence human flavor
perception when the participants were aware of the lack of any meaningful correspondence between
the color and the flavor of the solutions that they were tasting. The participants in Zampini et al.s
study were presented with strawberry, lime, orange fruitflavored solutions or flavorless solutions,
and requested to identify the flavor of each solution. Each of the different flavors was associated
equiprobably with each of the different colors (red, green, orange, and colorless). This meant that,
for example, the strawberry-flavored solutions were just as likely to be colored red, green, orange,
as to be presented as a colorless solution. Therefore, each of the solutions might have been colored
either appropriately or inappropriately (consisting of incongruently colored or colorless solutions). The participants were informed that they would often be tricked by the color of the solutions
that would often not correspond to the flavor typically associated with that color.
The most important finding to emerge from Zampini et al.s (2007) study was that color information has a strong impact on flavor identification even when participants were informed that the
colors of the drinks that they were testing were often misleading. In particular, flavors associated with appropriate colors (e.g., lime flavorgreen color; orange flavororange color) or colorless
were recognized far more accurately than when they were presented with an inappropriate coloring (i.e., lime-flavored drinks that were colored either red or orange; orange-flavored drinks that
were colored either green or red). These results therefore show that inappropriate coloring tends to
lead to impaired flavor discrimination responses, whereas appropriate coloring does not necessarily
improve the accuracy of participants flavor discrimination responses (at least when compared to
the flavor discrimination accuracy for the colorless solutions). Interestingly, however, no significant
effect of color was shown for the strawberry-flavored solutions. That is, the inappropriate coloring
of the strawberry-flavored solutions (i.e., when those solutions were colored green or orange) did not
result in a significant reduction in the participants ability to recognize the actual strawberry flavor.
One possible explanation for this result is that those flavors that are more strongly associated with
a particular color are more difficult to identify when presented in inappropriately colored solutions
(see Shankar et al. 2009). In fact, Zampini et al. (2007, Experiment 1; see Table 37.1) has shown that
the link between color and a specific flavor was stronger for the orange- and green-colored solutions
than for the red-colored solutions. That is, the participants in their study more often matched the
orange color with the flavor of orange and the green color with the flavor of lime. By contrast, the
red color was associated with strawberry, raspberry, and cherry flavors.
Whatever the reason for the difference in the effect of the various colors on participants flavor
discrimination responses, it is important to note that Zampini et al.s (2007) results nevertheless show
that people can still be misled by the inappropriate coloring of a solution even if they know that the
color does not provide a reliable guide to the flavor of the solution. By contrast, the participants in
743
TABLE 37.1
Flavors Most Frequently Associated with Each Colored Solution
in Zampini et al.s (2007, Experiment 1) Study
Color
Green
Orange
Yellow
Blue
Gray
Red
Colorless
Most Associated Flavors

Lime (69%)a
Orange (91%)a
Lemon (89%)a
Spearmint (86%)a
Black currant (53%), licorice (40%)a
Strawberry (46%), raspberry (27%), cherry (27%)
Flavorless (51%)a
Source: Zampini, M. et al., Food Qual. Prefer., 18, 975984, 2007. With permission.
a Significant colorflavor association tested using 2 analysis.
the majority of previous studies in this area (e.g., DuBose et al. 1980; Johnson and Clydesdale 1982;
Morrot et al. 2001; Oram et al. 1995; Philipsen et al. 1995; Roth et al. 1988; Stillman 1993; Zellner
and Durlach 2003) were not explicitly informed that the flavors of the solutions might not be paired
with the appropriately colored solutions. Zampini et al.s results therefore suggest that the modulatory role of visual information on multisensory flavor perception is robust enough to override any
awareness that participants might have (e.g., as informed by the experimenter) concerning the lack
of congruency between the color and the flavor of the solutions that they taste. However, it would be
interesting in future research to investigate whether knowing that there is no meaningful relationship between the color of the solutions and their flavor would modulate (i.e., reduce vs. enhance) the
influence of colors on flavors perception, as compared to the situation in which the participants are
not given any prior information about whether the colors are meaningfully related to the flavors.
37.2.2 Color-Flavor Interactions: Possible Role of Taster Status

Given recent interest in the consequences of individual differences in taster status on flavor perception (see Drewnowski 2003, for a review), Zampini and his colleagues (2008) wanted to investigate
whether any possible multisensory effects of visual (i.e., the colors of the solutions) and/or gustatory
(i.e., the presence vs. absence of fruit acids) cues on flavor perception might be affected by the taster
status of their participants. Previous research has demonstrated the existence of three subgroups
of tasters (nontasters, medium tasters, and supertasters), varying in their sensitivity to 6-n-propyl
thiouracil (PROP; e.g., Bartoshuk et al. 1992) as well as to a variety of other tastants (e.g., Prescott
et al. 2001; Reed 2008).* Surprisingly, however, none of the previous studies that has investigated
individual differences in taste perception has as yet looked at the possible influence of taster status
on the visual modulation of (or dominance over) flavor perception.
In Zampini et al.s (2008) study, the taster status of each participant was initially assessed using
suprathreshold PROP filter paper strips (see Bartoshuk et al. 1994). The participants had to place
the PROP filter paper strips on their tongue and then rate the intensity of the sensation of bitterness
* The individual differences in taste sensitivity most extensively studied are those for the bitterness intensity of PROP [and
phenylthiocarbamide (PTC) in earlier work]. Supertasters, medium tasters, and nontasters rate the bitterness of PROP as
very to intensely strong, moderate to strong, and weak, respectively. Research using taste solutions have identified other
differences in the three taster groups (see Prescott et al. 2004). Different PROP taster groups reported different taste
intensities and liking of other bitter, salty, sweet, and fat-containing substances. The three different PROP taster groups
are known to possess corresponding genetic differences. In particular, studies of taste genetics have revealed the exis
tence of multiple bitterness receptor genes (Kim et al. 2004; see also Bufe et al. 2005; Duffy 2007).
that they experienced on a Labelled Magnitude Scale (e.g., Green et al. 1993). The participants were
then classified into one of three taster groups: nontasters, medium tasters, and supertasters based
on the cutoff values (non-tasters <10.90; 10.91 < medium tasters < 61.48; supertasters > 61.49; see
also Essick et al. 2003, for a similar criterion). Zampini et al.s findings revealed that the modulatory
cross-modal effect of visual cues on peoples flavor identification responses were significantly more
pronounced in the nontasters than in the medium tasters, who, in turn, were influenced to a greater
extent by visual cues on their flavor identification responses than were the supertasters (see Figure
37.1). In particular, the nontasters (and, to a lesser extent, medium tasters) identified the flavors of
the solutions significantly more accurately when they were colored appropriately than when they
were colored inappropriately (or else were presented as colorless solutions). By contrast, the supertasters identified the flavors of the solutions more accurately overall, and their performance was not
affected by the colors of the solutions.
Zampini et al.s (2008) results are consistent with recent accounts of sensory dominance derived
from studies of cross-modal interactions between tactile, visual, and auditory stimuli (see, e.g.,
Alais and Burr 2004; Ernst and Banks 2002). Ernst and Banks used the maximum likelihood
Black currant
Yellow Gray Orange Red Colorless
75
50
25
0
Color of the solutions
Correct responses (%)
Medium
tasters
100
75
50
25
0
Supertasters
75
50
25
0
75
50
25
0
75
50
25
0
75
50
25
0
100
100
75
50
25
0
100
100
100
25
100
50
Flavorless
100
75
Orange
Nontasters
100
744
75
50
25
0
FIGURE 37.1 Mean flavor intensity ratings for three groups of participants (nontasters, medium tasters, and
supertasters) for blackcurrant, orange, and flavorless solutions presented in Zampini et al.s (2008) study of
effects of color cues on multisensory flavor perception in humans. Black columns represent solutions where
fruit acids had been added and white columns represent solutions without fruit acids. Error bars represent
between-participants standard errors of the means. (Reprinted from Zampini, M. et al., Food Qual. Prefer.,
18, 975984, 2007. With permission.)
745
estimation (MLE) approach to argue that the contribution of a given sensory input to multisensory
perception is determined by weighting the sensor estimates in each sensory modality by the noise
(or variance) present in that modality. It could be argued that in Zampini et al.s study, the estimates
of the flavors of the fruit-flavored solutions by the nontasters were simply more variable (i.e., their
judgments were less sensitive) than those of either the medium tasters or the supertasters. As a consequence, given the presumably uniform levels of visual discriminability across these three groups
of participants, the MLE account would predict that nontasters should weigh the visual cues more
highly when making their responses than the medium tasters, who in turn should weigh the gustatory cues less highly than the supertasters, just as we observed. It will be an interesting question
for future research to determine whether flavor discrimination responses can be modeled using the
MLE approach. It is important to note here that such an analysis may also be able to reveal whether
there are any underlying attentional biases (to weight information from one sensory modality more
highly than information from another modality) that may be present in the different taster groups
(cf. Battaglia et al. 2003). Moreover, it is interesting to consider at this point that although more than
100 studies examining visual contributions to flavor perception have been published over the past 80
years, Zampini et al.s study represents the first attempt to take the taster status of participants into
consideration when analyzing their results. The results of Zampini et al.s study clearly demonstrate
that taster status plays an important role in modulating the cross-modal contribution of visual cues
to flavor perception in fruit-flavored beverages.*
37.2.3 C
olorFlavor Interactions: Possible Role of Learned
Associations between Colors and Flavors
The influence of color on flavor perception may (and, some might say, must) be due to learned
associations between specific colors and particular flavors. Some of these associations are fairly
universal. For example, the flavor and color association of ripe fruits (Maga 1974; see also Morrot
et al. 2001). By contrast, other colorflavor association might be context dependent and so might
be different in different parts of the world (see Duncker 1939; Lucchelli et al. 1978; Shankar et al.
2010; Spence 2002; Wheatley 1973). For instance, lemons are typically yellow in Europe, whereas
in Colombia they are mostly dark green. Therefore, a particular colorflavor pairing that seems
congruent to people in a certain part of the world may seem incongruent to those who live elsewhere
(cf. Dematt et al. 2006, 2009).
Seventy years ago, Duncker (1939) considered the role of individual differences in learning such
associations. The participants in his early study were presented with milk chocolate that had been
colored brown or white (a new color for chocolate at the time the study was conducted) both with
the same flavor. Participants who had never seen white chocolate before reported that the white
chocolate had a different flavor to the brown-colored chocolate. The only participant who had come
across white chocolate before taking part in the study reported that the different colored chocolates
all tasted the same. Although it should be noted that this early study had a number of methodological limitations (i.e., only a small number of participants were tested, not to mention the fact that no
statistical analysis of the data was reported), the results nevertheless highlight the possible importance of prior experience and knowledge in modulation colorflavor interactions.
A follow-up of Dunckers (1939) seminal study has been conducted recently by Levitan et al.
(2008; see also Shankar et al. 2009). The researchers in this study investigated whether peoples
prior beliefs concerning specific colorflavor associations might not affect their ability to discriminate the flavor of colored sugar-coated chocolate sweets, Smarties (Nestl). Smarties are readily
available in eight different colors but only two different flavors, as test stimuli: the orange Smarties
* However, it should also be noted that the relatively small number of participants were tested in each category (i.e., four
non-tasters, five medium tasters, and five supertasters), thus placing a caveat in terms of generalizing from Zampini et
al.s (2008) findings. In future studies, taster status should therefore be assessed with much larger sample sizes.
746
that are produced for the UK market contain orange-flavored chocolate, whereas all of the other
colors contain unadulterated milk chocolate. By contrast, Smarties that have been produced for
other markets all contain unadulterated milk chocolate, regardless of their color. Crucially, the participants were sometimes presented with pairs of stimuli that differed in their color but not in their
flavor, or with pairs of Smarties that differed in both their color and flavor, or else with Smarties
pairs that differed in their flavor but not their color.
In a preliminary questionnaire, a number of the participants in Levitan et al.s (2008) study
stated their belief that a certain non-orange (i.e., red and green) Smartie had a distinctive flavor
(which is incorrect), whereas other participants believed (correctly) that all the non-orange Smarties
tasted the same. In the first experiment, the participants were presented with all possible pairings of
orange, red, and green Smarties and were asked to judge whether a given pair of Smarties differed
in flavor by tasting them while either sighted or blindfolded. The results showed that peoples beliefs
concerning specific colorflavor associations for Smarties exerted a significant modulatory effect
on their flavor responses. In the sighted condition, those participants who believed that non-orange
Smarties all taste the same were more likely to judge correctly that a redgreen pairing of Smarties
tasted identical in comparison to the first group, who performed at a level that was significantly
below chance (i.e., they reported that the red and green Smarties tasted different on the majority
of trials). In other words, those participants who thought that there was a difference between the
flavors of the red and green Smarties did in fact judge the two Smarties as tasting different far more
frequently when compared with participants who did not hold such a belief in the sighted condition.
The results of Levitan et al.s study are consistent with the results of the other studies presented in
this section in showing that food color can have a powerful cross-modal influence on peoples perception of the flavor of food. However, Levitan et al.s findings show that peoples beliefs about the
cross-modal colorflavor associations of specific foods can modulate this influence, and that such
cognitive influences can be robust and long-lasting despite extensive experience with the particular
food item concerned.*
In another recent study, Shankar et al. (2009) found that another variety of sugar-coated chocolate candies (multicolored M&Ms, which are all physically identical in taste) were rated as having a
stronger chocolate flavor when they were labeled as dark chocolate than when they were labeled
as milk chocolate. Many other studies have found a similar effect of expectations produced by
labeling a stimulus before sampling on flavor perception (see Cardello 1994; Deliza and MacFie
1996; Lee et al. 2006; Yeomans et al. 2008; Zellner et al. 2004, for reviews). Shankar et al. have
also investigated whether the influence of expectations on flavor perception might be driven by color
information (see Levitan et al. 2008). In their study, participants were asked to evaluate how chocolatey they found green- or brown-colored M&Ms. Participants rated the brown M&Ms as being
more chocolatey than the green ones. This result suggests that the color brown generates stronger
expectations of chocolate than the green color (cf. Duncker 1939). Finally, Shankar et al. studied
whether there was an interaction between the expectation generated by either color or label on multisensory flavor perception. The participants were again presented with brown- or green-colored
M&Ms and informed about the chocolate category (i.e., either milk chocolate or dark chocolate) with each colorlabel combination (greenmilk, brownmilk, greendark, browndark)
presented in a randomized order. Brown-colored M&Ms were given a higher chocolatey rating
than green-colored M&Ms. Similarly, those labeled as dark chocolate were given higher ratings
than those labeled milk chocolate. However, no interaction between these colors and labels was
found, thus suggesting that these two factors exerted independent effects, implying that two distinct
associations were being retrieved from memory and then utilized (e.g., the colorflavor association
* It is interesting to note that the participants in Levitan et al.s (2008) study were able to maintain such inappropriate
beliefs about differently colored Smarties tasting different, despite the objective evidence that people perceive no difference in their flavor, and the fact that they have presumably had extensive previous exposure to the fact that these colors
provide no useful information in this foodstuff.
747
and the labelflavor association). Shankar et al.s findings therefore provide the first evidence that
color can influence the flavor of a product whose flavor identity cannot be predicted by its color. In
other words, the colors of the coatings of the M&Ms are independent of their taste (which is always
chocolate).
One final issue that remains unresolved here concerns the extent to which the influence of color
on flavor discrimination reflects a perceptual versus a more decisional effect, or whether instead
both perceptual and decisional factors may contribute to participants performance (see Spence et
al., submitted; and Zampini et al. 2007, on this point). If it is a purely perceptual effect, the participants gustatory experience should be changed by viewing the color, that is, knowledge of the
color might improve the sensitivity of participants flavor discrimination responses by reducing the
variability of the multisensory flavor signal. Alternatively, however, according to the decisional
account, people should always have given the same gustatory response for a given colorflavor pairing regardless of whether sighted or blindfolded. In fact, what may have changed is their decisional
criteria. In Levitan et al.s (2008) study, the participants who were uncertain of their responses for
a given pair of Smarties might have biased their choice toward making different responses because
they could see that they had a different color. By contrast, those participants who already knew that
red and green Smarties were normally identical in taste might have been biased toward making a
same response. In the case of olfaction, Engen (1972) has already shown results consistent with the
claim that color can influence odor perception as a result of its effect on decisional mechanisms, but
this does not, of course, necessarily rule out a role for perceptual interactions as well, at least when
tested under the appropriate experimental conditions (see Zellner and Kautz 1990).
However, it is possible to hypothesize that a persons beliefs about particular foods tasting different if they have a different color may paradoxically result in them actually tasting different.
Analogously, de Craen et al. (1996) discussed a number of findings showing that color cues modulate the effectiveness of medicines as well as placebo pills. Although the mechanism behind placebo
effects such as these is not as yet well understood, the effects themselves are nevertheless robust
(e.g., for a recent review, see Koshi and Short 2007). What is more, just as in Levitan et al.s (2008)
Smarties experiment, there is at least some evidence that different people may hold different beliefs
about differently colored pills, and that these beliefs can carry over into the actual effects that the
differently colored placebo pills are shown to have (Lucchelli et al. 1978). Therefore, if peoples
beliefs about color and medication can affect their physical state (e.g., resulting in a genuine change
in their tolerance for pain, say, or in their ability to sleep), it would seem conceivable that a persons
belief that a certain colored Smartie tasted distinctive (from a Smartie of a different color) might
give rise to the effect of it, paradoxically, actually tasting different to that person, despite there being
no physical difference in flavor.
37.2.4 ColorFlavor Interactions: Neural Correlates

The results discussed so far on the potential influences of visual cues on flavor perception are consistent with the growing body of neurophysiological and electrophysiological data demonstrating
the intimate link between visual, olfactory, and gustatory flavor information at a neuronal level
(Osterbauer et al. 2005; Rolls 2004; Rolls and Baylis 1994; Small 2004; Small and Prescott 2005;
Verhagen and Engelen 2006). For instance, Osterbauer and his colleagues have used functional
neuroimaging to investigate how activity in the human orbitofrontal cortex (OFC) can be modulated by the presentation of particular combinations of odors and colors. The participants in this
study had to smell different odors including lemon, strawberry, spearmint, and caramel that were
presented by means of a computer-controlled olfactometer. The odors were presented in isolation or
else together with a color. The participants wore prism glasses to see full-screen colors presented
onscreen outside the magnet bore. On some occasions the odor matched the color, such as when
the smell of lemon was presented with the color yellow, whereas at other times the odor and color
did not match, such as when spearmint odor was presented with the color brown. Osterbauer et al.s
748
findings revealed that the presentation of appropriate odorcolor combinations (e.g., odor of strawberry matched with red color) increased the brain activity seen in the OFC when compared with the
brain activation seen in the odor-alone conditions. By contrast, there was a suppression of neural
activity in the same area when inappropriate colorodor combinations were presented (e.g., when
the odor of strawberry was presented with a turquoise patch of color on the monitor; see also De
Araujo et al. 2003). Taken together, these results would appear to suggest that presenting an appropriate colorodor association may actually lead to increased neural activity in brain areas responsible for processing olfactory stimuli, whereas presenting inappropriate colorodor associations can
suppress brain activity below that observed to the odors alone. The positive correlation between the
perceived congruency of colorodor pairs and the changes in the pattern of brain activation found
in Osterbauer et al.s study (see also Skrandies and Reuther 2008), therefore, provides a neurophysiological basis for the perceptual changes elicited by changing the color of food.
37.2.5 Interim Summary

Taken together, the results reviewed thus far demonstrate that visual information can have a dramatic impact on flavor perception and evaluation in humans. In particular, most of the studies have
shown that it is possible to impair flavor discrimination performance by coloring fruit-flavored
solutions inappropriately. The effect of color cues on human flavor perception can be explained by
the fact that visual information sets up an expectation regarding the flavor that is about to be experienced. This expectation may originate from any previous experience with similar food stimuli that
have contributed to build up such associations between the visual aspect and the experienced flavor
(see Shankar et al. 2010; Yeomans et al. 2008). Stevenson and his colleagues (e.g., Stevenson and
Boakes 2004; Stevenson et al. 1998) have suggested than any interaction taking place between gustation and olfaction might be explained in terms of associative learning processes. Their findings
show that we are able to create strong links between odors and tastes that are repeatedly presented
together. It is possible to hypothesize that the strong correspondences between colors and flavors
may rely on a similar mechanism. The same foodstuffs are usually experienced first through their
visual appearance and then through their flavor. It is possible that in our life we learn to build up
a strong association between visual and flavor food properties that are systematically combined.
Therefore, people who are presented first with the visual aspect of food and drinks generated a
series of expectations about the flavor that those food and drinks should have. White and Prescott
(2007) have put forward a similar explanation for their findings regarding the influence of odors on
tastes identification, when odors were presented in advance of taste.
In the previous section, a study was discussed in which participants beliefs on the colorflavor
association based on their previous experiences significantly modulated their responses (see Levitan
et al. 2008). In particular, participants who expected a difference between food products that were
colored differently were more likely to report a difference than those without any such prior belief.
Therefore, flavor perception might be considered as constituting a multisensory experience with
somewhat different rules that those regulating other multisensory interactions. Research suggests
that spatial coincidence and temporal synchrony are two of the key factors determining whether
multisensory integration will take place (at the single cell level) to give rise to the rich multisensory
perceptual objects that fill our everyday lives (for reviews, see Calvert et al. 2004). Given that the
cross-modal influence of visual cues on flavor perception occur long before we taste foods and occur
in different regions of space (i.e., food is only ever seen outside the oral cavity but tasted within it;
see Hutchings 1977), it would seem reasonable to suggest that expectancy plays a greater role than
the spatial and temporal rules (see Shankar et al. 2010). It might, for example, be less likely that
visualflavor interactions would be influenced by the spatial and temporal rules of multisensory
integration (which might better help to explain the integration of auditory, visual, and tactile, that is,
the spatial senses; it might also explain the integration of olfactory/gustatory and oralsomatosensory cues in the basic flavor percept). Therefore, we believe that the multisensory study of flavor
749
perception is particularly interesting for multisensory researchers precisely because the rules of
integration, and cross-modal influence, are likely to be somewhat different.
In the previous sections, we also discussed how individual differences can affect the nature
of the cross-modal visualflavor interactions that are observed. In particular, visual influences on
multisensory flavor perception can be significantly modulated as a function of the taster status of
the participant. Visual dominance effects in multisensory flavor perception are more pronounced
in those participants who are less sensitive to gustatory cues (i.e., nontasters) than in supertasters,
who appear to enjoy the benefit of enhanced gustatory resolution. Therefore, taster status, although
often neglected in studies investigating colorflavor interactions, should certainly be considered
more carefully in any future research in this area. Finally, we have reviewed the role of expectancy
resulting from visual information on the overall food perception.
37.3 R
OLE OF AUDITORY CUES IN THE MULTISENSORY
EXPERIENCE OF FOODSTUFFS
Most of the visual cues typically occur before our consumption of food and drink, whereas auditory
cues are typically only available at the moment of consumption (or mastication). Therefore, one
might expect the role of expectancy to be reduced when looking at the effect of sounds on the perception of food. Certainly, visual and auditory cues provide information at distinct stages of eating.
In the second part of this chapter, we therefore briefly discuss the possible role that auditory cues
may play in the multisensory perception of foodstuffs.
Several studies have demonstrated the influential role that auditory information plays in our
perception of food (for a review, see Spence and Zampini 2006). For example, it has been shown
that peoples ratings of the pleasantness of many foods can be strongly influenced by the sounds
produced when people bite into them (e.g., Drake 1970; Vickers 1981, 1983; Vickers and Bourne
1976). Food sounds have a particularly noticeable influence on peoples perception of crispness that
is closely associated with pleasantness, especially in crunchy foods (i.e., crisps; e.g., Vickers 1983).
Taken together, these results therefore suggest that the perception of the crispness of (especially)
crunchy foods (e.g., crisps, biscuits, cereals, vegetables) is largely characterized by tactile, mechanical, kinesthetic, and auditory properties (e.g., Vickers 1987).
Many foodstuffs produce particular sounds when we eat them. For instance, Drake (1963)
reported that the sounds produced by chewing or crushing a variety of different foodstuffs varied
in their amplitude, frequency, and temporal characteristics. Analysis of the auditory characteristics
of different foods has shown that crispy foods are typically higher in pitch than crunchy foods
(Vickers 1979). However, the role of auditory cues in the evaluation of food qualities (e.g., crispness) have been investigated by using different kinds of foodstuffs, that might have different levels
of freshness (e.g., Christensen and Vickers 1981; Drake 1963; Seymour and Hamann 1988; Vickers
1984; Vickers and Bourne 1976; Vickers and Wasserman 1979). Those studies also clearly show
that despite the informational richness contained in the auditory feedback provided by biting into
and/or chewing food, people are typically unaware of the effect that such sounds have on theiroverallmultisensory perception or evaluation of particular stimuli. In particular, Zampini and Spence
(2004, 2005) have shown that peoples perception and evaluation of different foodstuffs (e.g., potato
chips and sparkling water) can be modulated by changing the overall sound level or just the highfrequency components (see also Chen et al. 2005; Masuda et al. 2008; Varela et al. 2006).
37.3.1 Effect of Sound Manipulation on the Perception of Crisps

Zampini and Spence (2004) studied the multisensory interactions between auditory, oral, tactile,
mechanical, kinesthetic, and visual information in the rating of the perception of the crispness
and freshness of potato chips (or crisps), to investigate whether the evaluation of the crispness and
freshness of potato chips would be affected by only modifying the sounds produced during the biting action. In fact, the Pringles potato chips used in their experiment have all the same visual (i.e.,
shape) and oraltactile (i.e., texture) aspects. The participants in this study had to make a single
bite with their front teeth into a large number (180) of potato chips (otherwise known as crisps in
the United Kingdom) with their mouth placed directly above the microphone and then to spit the
crisp out (without swallowing) into a bowl placed on their lap. They then rated the crispness and
freshness of each potato chip using a computer-based visual analog scale. The participants might
hear the veridical sounds they made when biting into a crisp without any auditory frequency adjustment or with frequencies in the range 220 of the biting sounds amplified or attenuated by 12 dB.
Furthermore, for each frequency manipulation, there was an attenuation of the overall volume of
0 (i.e., no attenuation), 20, or 40 dB. The results showed that the perception of both crispness and
freshness were affected by the modulation of the auditory cues produced during the biting action. In
particular, the potato chips were perceived as being both crisper and fresher when either the overall
sound level was increased, or when just the high frequency sounds (in the range of 220 kHz) were
selectively amplified (see Figure 37.2).
(b)
(a)
Response scale
Headphones
Perceived crispness
(magnitude estimation)
Crisper 100
Softer
Microphone
0 dB
20 dB
80
60
40 dB
40
20
Attenuate
Normal
Amplify
Frequency manipulation
(c)
Fresher 100
Perceived freshness
(magnitude estimation)
750
Response pedals
Staler
0 dB
80
20 dB
60
40 dB
40
20
0
Attenuate
Normal
Amplify
Frequency manipulation
FIGURE 37.2 (a) Schematic view of apparatus and participant in Zampini et al.s (2004) study. Door of experimental booth was closed during the experiment and response scale was viewed through the window in left-hand
side wall of booth. Mean responses for softcrisp (b) and freshstale (c) response scales for three overall attenuation
levels (0, 20, or 40 dB) against three frequency manipulations (high frequencies attenuated, veridical auditory
feedback, or high frequencies amplified) are reported. Error bars represent between-participants standard errors of
means. (Reprinted from Zampini, M., and Spence, C., J. of Sens. Stud., 19, 347363, 2004. With permission.)
751
Given that the crisps in Zampini and Spences (2004) study were very similar to each other in
terms of their visual, tactile, and flavor attributes, the only perceptual aspect that varied during the
task was the sound (which, of course, also contributes to flavor). Therefore, participants may have
felt that the crisps had a different texture only guided by the sound since the other senses always
received the same information. Additional evidence highlighting the powerful effect of auditory
cues on the overall perception of the crisps was that the majority of the participants (15 out of 20)
stated anecdotally on debriefing after the experiment that they believed the crisps to have been
selected from different packages. Additionally, the majority of the participants also reported that
the auditory information had been more salient than the oral tactile information, and this may also
help to account for the effects reported by Zampini and Spence. In fact, one of the fundamental laws
of multisensory integration that has emerged over the past few decades states that the sense that
provides the more reliable (or salient) information is the one that dominates, or modulates, perception in another sensory modality (e.g., Ernst and Banks 2002; Shimojo and Shams 2001; Welch and
Warren 1980). However, the sensory dominance effect can be explained by the fact that the human
brain might rely on the most attended senses (Spence and Shankar 2010). The role of attention in the
multisensory influence of auditory information on food perception is consistent with the results of a
study in which the participants had to try and detect weak solutions of sucrose or citric acid in a mixture (Marks and Wheeler 1998). Participants were more accurate at detecting the tastant they were
attending to than for the tastant they were not attending to (see also Ashkenazi and Marks 2004).
Marks and Wheeler suggested that our ability to detect a particular sensory quality (e.g., tastant or
flavor) may be modulated by selective attention toward (or away from) that quality. Therefore, in a
similar vein, one might suggest that the effect found in crispness perception by increasing the overall loudness of the sounds produced when biting into crisps can change a participants perception
by making the sounds more pronounced than would have been the case if this information had been
derived solely from the texture in the mouth or from normal-level auditory cues. That is, participants attention would be directed toward this feature of the food by externally changing the relative
weighting of the sensory cues that signify this. Louder sounds are also presumably more likely to
capture a persons attention than quieter sounds. However, at present, it is unclear how many of the
findings taken to support an attentional account of any sensory dominance effect can, in fact, be
better accounted for in terms of sensory estimates of stimulus attributes simply being more accurate (i.e., less variable) in the dominant modality than those in the other modalities (e.g., Alais and
Burr 2004; Battaglia et al. 2003; Ernst and Banks 2002). Finally, it is important to note that these
explanations are not mutually exclusive. For example, Zampini and Spences (2004) results can be
accounted for either in terms of attentional capture or in terms of multisensory integration.
37.3.2 Effect of Auditory Cues on the Perception of Sparkling Water

In a follow-up study, Zampini and Spence (2005) studied the possible influence of auditory cues
in the perception and evaluation of carbonation of water. Our perception of the carbonation of
a beverage often relies on the integration of a variety of multisensory cues from visual, oral
somatosensory, nociceptive, auditory, and even tactile cues that are provided by the bubbles (e.g.,
Chandrashekar et al. 2009; Vickers 1991; Yau and McDaniel 1992). Zampini and Spence (2005)
examined the relationship between the auditory cues produced by sparkling water and its perceived
level of carbonation both when carbonated water samples were assessed in a cup and when they
were assessed in the mouth. The carbonation sounds were modified adopting the same experimental paradigm developed in their previous research on the perception of potato chips (Zampini and
Spence 2004). The sparkling water samples held in participants hands were judged to be more
carbonated whentheoverall sound level was increased and/or when the high-frequency components
(220 kHz) of the water sound were amplified. Interestingly, however, a subsequent experiment
failed to demonstrate any effect of these auditory manipulations on the perception of carbonation
and oral irritation from water samples that were held in the mouth. Taken together, these results
752
therefore show that auditory cues can modulate the perception of the carbonation of a water sample
held in the hand, but cannot modulate peoples perception of a water sample held in the mouth. This
might be because the perception of carbonation in the mouth is more dependent on oralsomatosensory and/or nociceptive inputs than on auditory cues, or alternatively, that it is more important that
we correctly perceive stimuli once they have entered the oral cavity (see Koza et al. 2005). Once
again, these findings are consistent with the hypothesis that the modality dominating multisensory
perception (when the senses are put into conflict) is the most accurate and/or informative sense (e.g.,
see Ernst and Banks 2002).
37.4 CONCLUSIONS
The past few years have seen a rapid growth of interest in the multisensory aspects of food perception
(see Auvray and Spence 2008; Delwiche 2004; Prescott 1999, 2004; Stevenson 2009; Stevenson and
Tomiczek 2007; Stillman 2002; Verhagen and Engelen 2006, for reviews). The research reviewed here
highlights the profound effect that visual (i.e., color of food) and auditory cues (i.e., variations in the
overall sound level and variations in the spectral distribution of energy) can have on peoples perception foodstuffs (such as potato chips and beverages). When people are asked to identify the flavors of
food and beverages, their responses can be influenced by the colors of those food and beverages. In
particular, the identification of specific flavors has often been shown to be less accurate when they are
paired with an inappropriate color (e.g., DuBose et al. 1980; Zampini et al. 2007, 2008). Our perception of the flavor and physical characteristics of food and beverages can also be modulated by auditory
cues. For instance, it is possible to change the perceived crispness of crisps or the perceived fizziness
of a carbonated beverage (such as sparkling water) simply by modifying the sounds produced when
eating the crisps or by the bubbles of the sparkling water (Zampini et al. 2004, 2005).
It is important to note that visual and auditory information are available at different stages of eating. Typically, visual (not to mention orthonasal olfactory and, on occasion, auditory) cues are available long before our ingestion of food (and before any other sensory cues associated with the food
are available). Therefore, visual cues (e.g., food colors) might be expected to create an expectancy
concerning the possible flavor of the food to be eaten (Hutchings 1977; Shankar et al. 2010). By
contrast, any role of expectancy might be reduced when thinking at the potential influence of auditory cues on the perception of food. In fact, the sounds produced when biting into or chewing food
are available at the moment of consumption. Therefore, it is possible to hypothesize that the role of
multisensory integration is somewhat different when looking at the role of visual and auditory cues
on the overall food perception. Given that visual cues are typically available long before a food is
consumed and outside the mouth, it is quite unlikely that visualflavor interactions are modulated
by the spatial and temporal rules (i.e., greater multisensory interaction with spatial and temporal
coincidence between the stimuli; see Calvert et al. 2004, for a review). Therefore, visual influences
on multisensory flavor perception are better explained by looking at the role of expectancy than at
the role of the spatial and temporal rules, which might help us to understand the role of auditory
cues on food perception instead. However, some sounds might produce an expectancy effect as well.
For example, sound of the food package being opened will normally precede the consumption of a
particular packaged food item (think only of the rattling of the crisps packet). Several researchers
have demonstrated that peoples expectations regarding what they are about to consume can also
have a significant effect on their perception of pleasantness of the food or drink itself (see Spence
et al., in press, for a recent review). It is also important to note that the visual and auditory contribution to multisensory flavor perception typically takes place without people necessarily being
consciously aware that what they are seeing or hearing is influencing their overall flavor experience
(e.g., Zampini et al. 2004, 2005). In Zampini et al.s more recent research (e.g., Zampini et al. 2007,
2008), the participants were influenced by the inappropriate colors of the beverages that they were
evaluating even though they had been informed beforehand that there might be a lack of congruency
between the colors that they saw and the flavors that they were tasting. This shows, therefore, that
753
the effect was powerful enough to override participants awareness that color information might
mislead their identification of the flavors. The potential role of the sounds made when eating food
on food perception is often ignored by people. For example, most of the participants in Zampini et
al.s (2004) study thought that the crisps were actually different (i.e., sorted from different packages or with different level of freshness and, therefore, of crispness). They seem to ignore the fact
that the experimenters changed only the sounds produced when biting into the crisps and the crisps
were not different. Nevertheless, the study reported here are consistent with a growing number of
neurophysiological and electrophysiological studies demonstrating close visualflavor (Osterbauer
et al. 2005; Small 2004; Small and Prescott 2006; Verhagen and Engelen 2006) and audiotacile
(Gobbel et al. 2003; Kitagawa and Spence 2006; Levnen et al. 1998; Schroeder et al. 2001; von
Bksy 1957)* interactions at the neuronal level. Results such as these therefore help to emphasize
the limitations that may be associated with relying solely on introspection and verbal report (as is
often the case in commercial consumer testing settings) when trying to measure peoples perception
and evaluation of foodstuffs.
REFERENCES
Alais, D., and D. Burr. 2004. The ventriloquist effect results from near-optimal bimodal integration. Current
Biology 14: 257262.
Alley, R. L., and T. R. Alley. 1998. The influence of physical state and color on perceived sweetness. Journal
of Psychology: Interdisciplinary and Applied 132: 561568.
Ashkenazi, A., and L. E. Marks. 2004. Effect of endogenous attention on detection of weak gustatory and olfactory flavors. Perception & Psychophysics 66: 596608.
Auvray, M., and C. Spence. 2008. The multisensory perception of flavor. Consciousness & Cognition 17:
10161031.
Bartoshuk, L. M., V. B. Duffy, and I. J. Miller. 1994. PTC/PROP tasting: Anatomy, psychophysics, and sex
effects. Physiology & Behavior 56: 11651171.
Bartoshuk, L. M., K. Fast, T. A. Karrer, S. Marino, R. A. Price, and D. A. Reed. 1992. PROP supertasters and
the perception of sweetness and bitterness. Chemical Senses 17: 594.
Battaglia, P. W., R. A. Jacobs, and R. N. Aslin. 2003. Bayesian integration of visual and auditory signals for
spatial localization. Journal of the Optical Society of America A 20: 13911397.
Bertelson, P., and G. Aschersleben. 1998. Automatic visual bias of perceived auditory location. Psychonomic
Bulletin & Review 5: 482489.
Blackwell, L. 1995. Visual clues and their effects on odour assessment. Nutrition and Food Science 5: 2428.
Bourne, M. C. 1982. Food texture and viscosity. New York: Academic Press.
Bufe, B., P. A. Breslin, C. Kuhn et al. 2005. The molecular basis of individual differences in phenylthiocarbamide and propylthiouracil bitterness perception. Current Biology 15: 322327.
Bult, J. H. F., R. A. de Wijk, and T. Hummel. 2007. Investigations on multimodal sensory integration: Texture,
taste, and ortho- and retronasal olfactory stimuli in concert. Neuroscience Letters 411: 610.
Cain, W. S. 1977. History of research on smell. In Handbook of perception: Vol. 6a: Tasting and smelling, ed.
E. C. Carterette and M. P. Friedman, 197229. New York: Academic Press.
Calvert, G., C. Spence, and B. E. Stein. 2004. The handbook of multisensory processing. Cambridge, MA: MIT
Press.
Cardello, A. V. 1994. Consumer expectations and their role in food acceptance. In Measurement of food
preferences, ed. H. J. H. MacFie, and D. M. H. Thomson, 253297. London: Blackie Academic &
Professional.
Chan, M. M., and C. Kane-Martinelli. 1997. The effect of color on perceived flavour intensity and acceptance
of foods by young adults and elderly adults. Journal of the American Dietetic Association 97: 657659.
Chandrashekar, J., D. Yarmolinsky, L. von Buchholtz et al. 2009. The taste of carbonation. Science 326:
443445.
* However, it is important to note that, to the best of our knowledge, no neuroimaging studies have as yet been conducted
to investigate the role of auditory cues on multisensory food perception (cf. Spence and Zampini 2006; Verhagen and
Engelen 2006).
754
Chen, H., C. Karlsson, and M. Povey. 2005. Acoustic envelope detector for crispness assessment of biscuits.
Journal of Texture Studies 36: 139156.
Christensen, C. M. 1980a. Effects of taste quality and intensity on oral perception of viscosity. Perception &
Psychophysics 28: 315320.
Christensen, C. M. 1980b. Effects of solution viscosity on perceived saltiness and sweetness. Perception &
Psychophysics 28: 347353.
Christensen, C. M., and Z. M. Vickers. 1981. Relationship of chewing sounds to judgments of food crispness.
Journal of Food Science 46: 574578.
Clydesdale, F. M. 1993. Color as a factor in food choice. Critical Reviews in Food Science and Nutrition 33:
83101.
Dalton, P., N. Doolittle, H. Nagata, and P. A. S. Breslin. 2000. The merging of the senses: Integration of subthreshold taste and smell. Nature Neuroscience 3: 431432.
Davis, R. 1981. The role of nonolfactory context cues in odor identification. Perception & Psychophysics 30:
8389.
De Araujo
, I. E. T., E. T.
Rolls,
M. L. Kringelbach, F. McGlone, and N. Phillips. 2003.

Tasteolfactory convergence, and the representation of the pleasantness of flavour, in the human brain. European Journal of
Neuroscience 18: 20592068.
de Craen, A. J. M., P. J. Roos, A. L. de Vries, and J. Kleijnen. 1996. Effect of colour of drugs: Systematic review
of perceived effect of drugs and their effectiveness. British Medical Journal 313: 16241626.
Deliza, R., and H. MacFie. 1996. The generation of sensory expectation by external cues and its effect on sensory perception and hedonic ratings: A review. Journal of Sensory Studies 11: 103128.
Delwiche, J. 2004. The impact of perceptual interactions on perceived flavour. Food Quality and Preference
15: 137146.
Dematt, M. L., D. Sanabria, and C. Spence. 2006. Crossmodal associations and interactions between olfaction
and vision. Chemical Senses 31: E50E51.
Dematt, M. L., D. Sanabria, and C. Spence. 2009. Olfactory identification: When vision matters? Chemical
Senses 34: 103109.
Drake, B. K. 1963. Food crunching sounds. An introductory study. Journal of Food Science 28: 233241.
Drake, B. K. 1970. Relationships of sounds and other vibrations to food acceptability. Proceedings of the 3rd
International Congress of Food Science and Technology, pp. 437445. August 914, Washington, DC.
Drewnowski, A. 2003. Genetics of human taste perception. In Human olfaction and gustation, 2nd ed., ed.
R.L. Doty, 847860. New York: Marcel Dekker, Inc.
DuBose, C. N., A. V. Cardello, and O. Maller. 1980. Effects of colourants and flavourants on identification,
perceived flavour intensity, and hedonic quality of fruit-flavoured beverages and cake. Journal of Food
Science 45: 13931399, 1415.
Duffy, V. B. 2007. Variation in oral sensation: Implications for diet and health. Current Opinion in
Gastroenterology 23: 171177.
Duncker, K. 1939. The influence of past experience upon perceptual properties. American Journal of Psychology
52: 255265.
Engen, T. 1972. The effect of expectation on judgments of odour. Acta Psychologica 36: 450458.
Ernst, M. O., and M. S. Banks. 2002. Humans integrate visual and haptic information in a statistically optimal
fashion. Nature 415: 429433.
Essick, G. K., A. Chopra, S. Guest, and F. McGlone. 2003. Lingual tactile acuity, taste perception, and the density and diameter of fungiform papillae in female subjects. Physiology & Behavior 80: 289302.
Frank, R. A., K. Ducheny, and S. J. S. Mize. 1989. Strawberry odor, but not red color, enhances the sweetness
of sucrose solutions. Chemical Senses 14: 371377.
Garber Jr., L. L., E. M. Hyatt, and . . Boya. 2008. The
mediating effects of the appearance of nondurable consumer goods and their packaging on consumer behavior. In Product experience, ed. H. N. J.
Schifferstein and P. Hekkert, 581602. London: Elsevier.
Garber Jr., L. L., E. M. Hyatt, and R. G. Starr Jr. 2000. The effects of food colour on perceived flavour. Journal
of Marketing Theory and Practice 8: 5972.
Garber Jr., L. L., E. M. Hyatt, and R. G. Starr Jr.
2001. Placing food color experimentation into a valid consumer context. Journal of Food Products Marketing 7: 324.
Gifford, S. R., and F. M. Clydesdale. 1986. The psychophysical relationship between colour and sodium chloride concentrations in model systems. Journal of Food Protection 49: 977982.
Gifford, S. R., F. M. Clydesdale, and R. A. Damon Jr. 1987. The psychophysical relationship between colour
and salt concentration in chicken flavoured broths. Journal of Sensory Studies 2: 137147.
755
Gobbel, R., M. Schrrmann, N. Forss, K. Juottonen, H. Buchner, and R. Hari. 2003. Activation of the human
posterior and tempoparietal cortices during audiotactile interaction. Neuroimage 20: 503511.
Green, B. G., G. S. Shaffer, and M. M. Gilmore. 1993. Derivation and evaluation of a semantic scale of oral
sensation with apparent ratio properties. Chemical Senses 18: 683702.
Hollowood, T. A., R. S. T. Linforth, and A. J. Taylor. 2002. The effect of viscosity on the perception of flavour.
Chemical Senses 27: 583591.
Hutchings, J. B. 1977. The importance of visual appearance of foods to the food processor and the consumer.
In Sensory properties of foods, ed. G. G. Birch, J. G. Brennan, and K. J. Parker, 4557. London: Applied
Science Publishers.
Johnson, J. L., and F. M. Clydesdale. 1982. Perceived sweetness and redness in coloured sucrose solutions.
Johnson, J. L., E. Dzendolet, and F. M. Clydesdale. 1983. Psychophysical relationships between sweetness and
redness in strawberry-drinks. Journal of Food Protection 46: 2125.
Kim, U. K., P. A. Breslin, D. Reed, and D. Drayna. 2004. Genetics of human taste perception. Journal of Dental
Research 83: 448453.
Kitagawa, N., and C. Spence. 2006. Audiotactile multisensory interactions in information processing. Japanese
Psychological Research 48: 158173.
Koshi, E. B., and C. A. Short. 2007. Placebo theory and its implications for research and clinical practice: A
review of the recent literature. Pain Practice 7: 420.
Koza, B., A. Cilmi, M. Dolese, and D. Zellner. 2005. Color enhances orthonasal olfactory intensity and reduces
retronasal olfactory intensity. Chemical Senses 30: 643649.
Lavin, J., and H. T. Lawless. 1998. Effects of colour and odor on judgments of sweetness among children and
adults. Food Quality and Preference 9: 283289.
Lawless, H., P. Rozin, and J. Shenker. 1985. Effects of oral capsaicin on gustatory, olfactory and irritant sensations and flavor identification in humans who regularly or rarely consume chili pepper. Chemical Senses
10: 57989.
Lee, L., S. Frederick, and D. Ariely. 2006. Try it, youll like it. Psychological Science 17: 10541058.
Levnen, S., V. Jousmki, and R. Hari. 1998. Vibration-induced auditory-cortex activation in a congenitally
deaf adult. Current Biology 8: 869872.
Levitan, C. A., M. Zampini, R. Li, and C. Spence. 2008. Assessing the role of colour cues and peoples beliefs
about colourflavour associations on the discrimination of the flavour of sugar-coated chocolates.
Chemical Senses 33: 415423.
Lucchelli, P. E., A. D. Cattaneo, and J. Zattoni. 1978. Effect of capsule colour and order of administration of
hypnotic treatments. European Journal of Clinical Pharmacology 13: 153155.
Maga, J. A. 1974. Influence of colour on taste thresholds. Chemical Senses and Flavour 1: 115119.
Marks, L. E., and M. E. Wheeler. 1998. Attention and the detectability of weak-taste stimuli. Chemical Senses
23: 1929.
Masuda, M., Y. Yamaguchi, K. Arai, and K. Okajima. 2008. Effect of auditory information on food recognition.
IEICE Technical Report 108(356): 123126.
Moir, H. C. 1936. Some observations on the appreciation of flavour in food stuffs. Chemistry and Industry 55:
145148.
Morrot, G., F. Brochet, and D. Dubourdieu. 2001. The colour of odors. Brain and Language 79: 309320.
Murphy, C., and W. S. Cain. 1980. Taste and olfaction: Independence vs. interaction. Physiology and Behavior
24: 601605.
Oram, N., D. G. Laing, I. Hutchinson et al. 1995. The influence of flavour and colour on drink identification by
children and adults. Developmental Psychobiology 28: 239246.
Osterbauer, R. A., P. M. Matthews, M. Jenkinson, C. F. Beckmann, P. C. Hansen, and G. A. Calvert. 2005. Color
of scents: Chromatic stimuli modulate odor responses in the human brain. Journal of Neurophysiology
93: 34343441.
Pangborn, R. M. 1960. Influence of colour on the discrimination of sweetness. American Journal of Psychology
73: 229238.
Pangborn, R. M., and B. Hansen. 1963. The influence of colour on discrimination of sweetness and sourness in
pear-nectar. American Journal of Psychology 76: 315317.
Philipsen, D. H., F. M. Clydesdale, R. W. Griffin, and P. Stern. 1995. Consumer age affects response sensory
characteristics of a cherry flavoured beverage. Journal of Food Science 60: 364368.
Prescott, J. 1999. Flavour as a psychological construct: Implications for perceiving and measuring the sensory
qualities of foods. Food Quality and Preference 10: 349356.
756
Prescott, J. 2004. Psychological processes in flavour perception. In Flavour perception, ed. A. J. Taylor and D.
Roberts, 256278. London: Blackwell Publishing.
Prescott, J., N. Ripandelli, and I. Wakeling. 2001. Binary taste mixture interactions in PROP non-tasters,
medium-tasters and super-tasters. Chemical Senses 26: 9931003.
Prescott, J., J. Soo, H. Campbell, and C. Roberts. 2004. Response of PROP taster groups to variations in sensory qualities within foods and beverages. Chemical Senses 26: 9931003.
Reed, D. R. 2008. Birth of a new breed of supertaster. Chemical Senses 33: 489491.
Rolls, E. T. 2004. Smell, taste, texture, and temperature multimodal representations in the brain, and their relevance to the control of appetite. Nutrition Reiews 62: S193S204.
Rolls, E. T., and L. L. Baylis. 1994. Gustatory, olfactory, and visual convergence within the primate orbitofrontal cortex. Journal of Neuroscience 14: 54375452.
Roth, H. A., L. J. Radle, S. R. Gifford, and F. M. Clydesdale. 1988. Psychophysical relationships between perceived
sweetness and colour in lemon- and lime-flavoured drinks. Journal of Food Science 53: 11161119, 1162.
Rozin, P. 1982. Tastesmell confusions and the duality of the olfactory sense. Perception & Psychophysics
31: 397401.
Schroeder, C. E., R. W. Lindsley, C. Specht, A. Marcovici, J. F. Smiley, and D. C. Javitt. 2001. Somatosensory in
put to auditory association cortex in the macaque monkey. Journal of Neurophysiology 85: 13221327.
Seymour, S. K., and D. D. Hamann. 1988. Crispness and crunchiness of selected low moisture foods. Journal
of Texture Studies 19: 7995.
Shankar, M. U., C. A. Levitan, J. Prescott, and C. Spence. 2009. The influence of color and label information
on flavor perception. Chemosensory Perception 2: 5358.
Shankar, M. U., C. Levitan, and C. Spence. 2010. Grape expectations: Does higher level knowledge mediates
the interpretation of multisensory cues? Consciousness & Cognition 19: 380390.
Shimojo, S., and L. Shams. 2001. Sensory modalities are not separate modalities: Plasticity and interactions.
Current Opinion in Neurobiology 11: 505509.
Skrandies, W., and N. Reuther. 2008. Match and mismatch of taste, odor, and color is reflected by electrical
activity in the human brain. Journal of Psychophysiology 22: 175184.
Small, D. M. 2004. Crossmodal integrationinsights from the chemical senses. Trends in Neurosciences 27:
120123.
Small, D. M., J. C. Gerber, Y. E. Mak, and T. Hummel. 2005. Differential neural responses evoked by orthonasal versus retronasal odorant perception in humans. Neuron 47: 593605.
Small, D. M., and J. Prescott 2005. Odor/taste integration and the perception of flavour. Experimental Brain
Research 166: 345357.
Small, D. M., M. G. Veldhuizen, J. Felsted, Y. E. Mak, and F. McGlone. 2008. Separable substrates for anticipatory and consummatory food chemosensation. Neuron 57: 786797.
Spence, C. 2002. The ICI report on the secret of the senses. London: The Communication Group.
Spence, C., C. Levitan, M. U. Shankar, and M. Zampini. 2010. Does food colour influence flavour identification
in humans? Chemosensory Perception 3: 6884.
Spence, C., and M. U. Shankar. 2010. The influence of auditory cues on the perception of, and responses to,
food and drink. Journal of Sensory Studies 25: 406430.
Spence, C., M. U. Shankar, and H. Blumenthal. In press. Sound bites: Auditory contributions to the perception and consumption of food and drink. To appear in Art and the senses, ed. F. Bacci and D. Melcher.
Oxford: Oxford Univ. Press.
Spence, C., and M. Zampini. 2006. Auditory contributions to multisensory product perception. Acta Acustica
united with Acustica 92: 10091025.
Stevenson, R. J., and R. A. Boakes. 2004. Sweet and sour smells: Learned synesthesia between the senses of
taste and smell. In The handbook of multisensory processes, ed. G.A. Calvert, C. Spence, and B. E. Stein,
6983. Cambridge, MA: MIT Press.
Stevenson, R. J., R. A. Boakes, and J. Prescott. 1998. Changes in odor sweetness resulting from implicit learning of a simultaneous odorsweetness association: An example of learned synesthesia. Learning and
Motivation 29: 113132.
Stevenson, R. J. 2009. The psychology of flavour. Oxford: Oxford Univ. Press.
Stevenson, R. J., and M. Oaten. 2008. The effect of appropriate and inappropriate stimulus color on odor discrimination. Perception & Psychophysics 70: 640646.
Stevenson, R. J., and C. Tomiczek. 2007. Olfactory-induced synesthesias: A review and model. Psychological
Bulletin 133: 294309.
Stillman, J. 1993. Colour influences flavour identification in fruit-flavoured beverages. Journal of Food Science
58: 810812.
757
Stillman, J. A. 2002. Gustation: Intersensory experience par excellence. Perception 31: 14911500.
Strugnell, C. 1997. Colour and its role in sweetness perception. Appetite 28: 85.
Tyle, P. 1993. Effect of size, shape and hardness of particles in suspension on oral texture and palatability. Acta
Psychologica 84: 111118.
Varela, P., J. Chen, C. Karlsson, and M. Povey. 2006. Crispness assessment of roasted almonds by an integrated
approach to texture description: Texture, acoustics, sensory and structure. Journal of Chemometrics 20:
311320.
Verhagen, J. V., and L. Engelen. 2006. The neurocognitive bases of human multimodal food perception: Sensory
integration. Neuroscience and Biobehavioral Reviews 30: 613650.
Vickers, Z. M. 1979. Crispness and crunchiness in foods. In Food texture and rheology, ed. P. Sherman, 145
166. London: Academic Press.
Vickers, Z. M. 1981. Relationships of chewing sounds to judgments of crispness, crunchiness and hardness.
Vickers, Z. M. 1983. Pleasantness of food sounds. Journal of Food Science 48: 783786.
Vickers, Z. M. 1984. Crispness and crunchinessA difference in pitch? Journal of Texture Studies 15:
157163.
Vickers, Z. M. 1987. Crispness and crunchinessTextural attributes with auditory components. In Food texture: Instrumental and sensory measurement, ed. H. R. Moskowitz, 4566. New York: Marcel Dekker.
Vickers, Z. M. 1991. Sound perception and food quality. Journal of Food Quality 14: 8796.
Vickers, Z. M., and M. C. Bourne. 1976. A psychoacoustical theory of crispness. Journal of Food Science 41:
11581164.
Vickers, Z. M., and S. S. Wasserman. 1979. Sensory qualities of food sounds based on individual perceptions.
Journal of Texture Studies 10: 319332.
von Bksy, G. 1957. Neural volleys and the similarity between some sensations produced by tones and by skin
vibrations. Journal of the Acoustical Society of America 29: 10591069.
Welch, R. B., and D. H. Warren. 1980. Immediate perceptual response to intersensory discrepancy. Psychological
Bulletin 3: 638667.
Wheatley, J. 1973. Putting colour into marketing. Marketing 67: 2429.
White, T. L., and J. Prescott. 2007. Chemosensory cross-modal Stroop effects: Congruent odors facilitate taste
identification. Chemical Senses 32: 337341
Yau, N. J. N., and M. R. McDaniel. 1992. The effect of temperature on carbonation perception. Chemical
Senses 14: 337348.
Yeomans, M., L. Chambers, H. Blumenthal, and A. Blake. 2008. The role of expectancy in sensory and hedonic
evaluation: The case of smoked salmon ice-cream. Food Quality and Preference 19: 565573.
Zampini, M., D. Sanabria, N. Phillips, and C. Spence. 2007. The multisensory perception of flavour: Assessing
the influence of colour cues on flavour discrimination responses. Food Quality & Preference 18:
975984.
Zampini, M., and C. Spence. 2004. The role of auditory cues in modulating the perceived crispness and staleness of potato chips. Journal of Sensory Studies 19: 347363.
Zampini, M., and C. Spence. 2005. Modifying the multisensory perception of a carbonated beverage using
auditory cues. Food Quality and Preference 16: 632641.
Zampini, M., E. Wantling, N. Phillips, and C. Spence. 2008. Multisensory flavour perception: Assessing the
influence of fruit acids and colour cues on the perception of fruit-flavoured beverages. Food Quality &
Preference 18: 335343.
Zellner, D. A., A. M. Bartoli, and R. Eckard. 1991. Influence of colour on odor identification and liking ratings.
American Journal of Psychology 104: 547561.
Zellner, D. A., and P. Durlach. 2003. Effect of colour on expected and experienced refreshment, intensity, and
liking of beverages. American Journal of Psychology 116: 633647.
Zellner, D. A., and M. A. Kautz. 1990. Colour affects perceived odor intensity. Journal of Experimental
Psychology: Human Perception and Performance 16: 391397.
Zellner, D. A., and L. A. Whitten. 1999. The effect of colour intensity and appropriateness on color-induced
odor enhancement. American Journal of Psychology 112: 585604.

Assessing The Role of Visual and Auditory Cues in Multisensory Perception of Flavor

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Assessing The Role of Visual and Auditory Cues in Multisensory Perception of Flavor

Enviado por

Direitos autorais:

Formatos disponíveis

the Role of

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

The Neural Bases of Multisensory Processes

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

The Neural Bases of Multisensory Processes

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

Most Associated Flavors

37.2.2 Color-Flavor Interactions: Possible Role of Taster Status

The Neural Bases of Multisensory Processes

Yellow Gray Orange Red Colorless

Color of the solutions

Correct responses (%)

Correct responses (%)

Yellow Gray Orange Red Colorless

Correct responses (%)

Yellow Gray Orange Red Colorless

Color of the solutions

Yellow Gray Orange Red Colorless

Color of the solutions

Color of the solutions

Yellow Gray Orange Red Colorless

Color of the solutions

Yellow Gray Orange Red Colorless

Color of the solutions

Yellow Gray Orange Red Colorless

Color of the solutions

Correct responses (%)

Correct responses (%)

Correct responses (%)

Correct responses (%)

Correct responses (%)

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

Yellow Gray Orange Red Colorless

Color of the solutions

Yellow Gray Orange Red Colorless

Color of the solutions

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

The Neural Bases of Multisensory Processes

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

37.2.4 ColorFlavor Interactions: Neural Correlates

The Neural Bases of Multisensory Processes

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

37.2.5 Interim Summary

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

37.3.1 Effect of Sound Manipulation on the Perception of Crisps

The Neural Bases of Multisensory Processes

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

37.3.2 Effect of Auditory Cues on the Perception of Sparkling Water

The Neural Bases of Multisensory Processes

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

The Neural Bases of Multisensory Processes

M. L. Kringelbach, F. McGlone, and N. Phillips. 2003.

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

The Neural Bases of Multisensory Processes

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

Downloaded by [University of Cincinnati] at 09:36 20 May 2016

Você também pode gostar