Film-Philosophy Conference 2013: Beyond Film

Mrcia A. Baldissera
Panel: Neuroaesthetics
Panel chair: Patricia Pisters
Location: University of Amsterdam, July 12, 2013, Amsterdam

The search for universals without falling into uniformity-conformity 1

In general, experiments on Neuroaesthetics try to find the universals in the relation between
the subjects and the objects to explain how, and sometimes why, the aesthetic experience occurs.
But, there is a clear tension between the richness of the aesthetic experience, due to its
subjectiveness, and the objective approach in neurosciences. How to proceed then into the study of
film reception within cognitive sciences? I propose an analysis of the methods, objects and
conclusions of some neuroscientific experiments with films to search for the conditions of the
aesthetic attention. First, I will show the pertinence of the concept of aesthetic attention. Then, I will
describe briefly the objet film and how it is used on these experiments. And finally, I will discuss
the methods and the conclusions of these experiments. My aim here is to highlight the principles of
the theories and the practices of film behind this tension.
The cinematographic experience is a relation between an specific film and an individual
with his own beliefs, goals and background knowledge in a determined context, with a proper
dynamics. This interaction starts with perception, which is caused by the flux of images, sounds and
linguistic signs of the film. If perception is the way of acquiring information from the stimuli, the
differences in the cognitive processing of these informations are due mainly to attention. Attention
is necessary to integrate the audiovisual mental representations with the processes of memory,
imagination, reasoning, of the single spectator within the given situation (Allport, 2011). So, the
attentional processing selects and enriches the mental representations, to give coherence, unity, to
our experience. That's why attention is directly related to the degree of consciousness of this
experience (Mole, 2009). And consequently, attention is related to our comprehension of this
experience and with its possible knowledge.
I would like to make two remarks. The first one is relate to the differences between an
attentional treatment that is stimulus- or exogenous-driven and an attentional treatment that is
endogenous-driven. Our perception is driven by the stimulus in a bottom-up manner (which is
automatic, preconscious, fast) and then, depending on the behavioural, semantic, and/or emotional
pertinence of the stimulus, we can orient our attention to this stimulus. Or, an internal process can
1

Theoriginaltitlewashomologation,whichhasnotexactlythesamesenseasthefrenchhomologation.Ithank
DeborahWhitworthforherprofessionaladvice.
1

drives our attention to the stimulus voluntarily. Thus, attention has to do with the breaches on the
subject's expectations (Corbetta et al., 2008), it has to do with the quality of the experience.
That lead us to the second remark: Hunt, Zoest and Kingstone (2010) points out that as much
as the subject has time, as much as the mental representation becomes more sophisticated,
incorporating high level background knowledge. The enrichment, or the integration, of the mental
representation by attention depends on time as well, that is, it depends on the phenomenological
structure and on the complexity of the stimulus; on the subject's goals and context; and on attention
time itself. So, attention has its temporal degrees, and it has its modes of inflection as well.
Consequently, our attention oscillates the whole period of the film.
What characterises an aesthetic experience is the aesthetic attention, a specific inflection of
our attention that can be tracked back to Kant's Critique of the Power of Judgement (Genette, 1999;
Schaeffer, 2013). It is not goal- or task-oriented, that is, the aesthetic attention does not have an
utilitarian or instrumental function, which means that our engagement to the film comes from our
own satisfaction with this experience. It is a pleasure, or a initial displeasure, on the process to
comprehend the object that causes the experience. So, we enjoy it even without a result (a
judgement for example) or a reward (like an emotional relief). This does not mean we can not make
a judgement or have an emotional relief, but this is not our objective. Empirically, this inflection of
attention seems to be related to activity in the default mode network (Corbetta et al., 2008).
The first consequence of this subjective function of the aesthetic attention is that initially we
are kind of story-oriented, but also in a kind of exploratory mode. This means that the attentional
processing of the mental representations is oriented, re-oriented, and re-oriented again, in a
continuos feedback with the film. Following this process, the mental representation's treatment
works more horizontally than vertically: that is, we favour the details and the associations, more
than the general categorisation. It's a matter of understanding more than meaning (Schaeffer, 2009).
For this reason, the attentional treatment is effortful, flexible and slow, instead of the economy
principle of the automatic treatment that goes straight to categorisation (Schaeffer, 2013). This
feedback treatment can be associated to the kantian free play of cognitive faculties, which can be
described by the Reverse Hierarchy Theory of Hochstein and Ahissar (2002).
Finally, this experience is an opportunity to play and to experiment in a context that is save
and free of the constraints of our day-by-day lives. It's an experience free of the survival or the
problem solving constraints. This process leads us to an experience of knowledge, a kind of
practical and social learning: a psychological, relational, emotional, moral know-how, more than a
propositional or a perceptive one (Schaeffer, 2010). Empirical data suggests that the temporoparietal
junction (TPJ), part of the default mode network, is active to both attentional and social cognition
2

situations (Corbetta et al., 2008).

So, attention is the key process in the interaction between the spectator and the film. What
about the objects, that is, the films, of some neuroscientific experiments? As we have seen, the
cinematographic flux has a causal relation to our perception, that is, the properties of every film
cause our perception. It is generally an audiovisual object, with a proper duration (of the single
shots and in total), with its own intra-relations: between images, between sounds, between image
and sound, etc. It is a fully construct Intentional representation (Searle, 1983), made by a person, or
a group of persons. In other words, films work as reception signs emitted by the authors. Films are
part of a larger context as well: human history, cinema history, cinema theories, etc.
But, in general, neuroscientific experiments using film have a deficit on the comprehension of
cinema: some take out the sound, and some show just a part of it. Most significant, some authors
simply ignore its history. That is, they ignore the large production and reception of many different
films around the world, over time, and the differences implied in this history. None of the presented
experiments respected the original form of the film (except for Germeys and d'Ydewalle, 2007, who
made one). Therefore, their conclusions are to be taken with caution.
We are dealing with a complex object and with a complex process as well. Attention can
manifest itself or be hidden. It is an interaction between our organism in a given situation. And, as
all the cognitive processes, attention lasts and changes in time. So, how can we analyse attention?
And what about the aesthetic attention?
The eye-tracker is a device not so invasive, that means it does not disturb so much the
subject's experience. It is used to identify the direction of the eye movements and the duration of
their fixations. An experiment made by Hristova and Grinberg (2011), that compared the eye gazes
of the participants when looking at a Dali and at a Caravaggio, confirms empirical results
reproduced over time. It demonstrates the relation between eye movements and attention : at the
beginning, the amplitudes are larger and the fixations are shorter, which seems to indicate a bottomup exploratory perception process. The amplitudes become smaller and the fixations longer when
perception seems focus attentionally.
Based on many experiments with the eye-tracker, Tim Smith (2012) defends an attentional
synchrony to the films following the continuity editing rule, because of the high-degree of
coordination of where multiple viewers attend in a dynamic scene and across
cuts (ibid, p. 21). The first objection is: from the research in perception, it's well known
that generally we look at the centre of the frame, to the faces, and are attract to movement and
speech, automatically. The second, Smith says nothing about the amplitudes and fixations of the eye
movements of the participants, which means that he does not present any empirical evidence to
3

argue for an attentional processing of these films 2, notably There Will Be Blood (Anderson, 2007),
Blade Runner (Scott, 1982), and Requiem for a Dream (Aronofsky, 2000).
Germeys and dYdewalle (2007), by the other hand, constructed a very simple video to
answer the question whether the violation of the rule of the 180 axis causes confusion or an
attempt to re-orient attention. This rule maintains continuity in the direction of an action, in a leftright basis. They analysed the dynamics of the eye movements and concluded that, not only the
violation does not disturb the viewers comprehension of the dialogue, but also, that It is indeed
common to observe increased fixation durations to a stimulus that violates expectations, as we
have already remarked. So, the violation of this continuity rule does not disrupt the cognitive
processing of the stimuli, and, it seems, it can even arousal attention.
The functional magnetic resonance imaging( fMRI) is a very invasive device, but it is the best
one for detecting the architecture of the cerebral activity, as it has a more spatial than temporal
resolution (its updating is about every second, in the best cases). This is one of the main critiques
towards fMRI, because when looking at the neural correlates of attention, the results show that
activity in the pre-frontal region is not sufficient, but the synchronicity of neurone groups, from a
short to a large brain scale, in the fraction of milliseconds (Allport, 2011). So, for the attentional
dynamics, electroencephalography (EEG), for example, is much more accurate.
The inter-subject correlation experiment made by Hasson and colleagues (2008b) with the
films The Good, the Bad and the Ugly (Leone, 1966), Bang! Youre Dead (Hitchcock, 1961), and an
episode of Davids Curb Your Enthusiasm (2000), concluded that some films can exert
considerable control over brain activity (ibid, abstract). But, following an attentive reading, only
the sensory regions presented a considerable inter-subject correlation, while activity in some median
regions can indicate only some multimodal integration of the mental representations, and there was
any activity in the pre-frontal regions, that could indicate an attentional processing. This experiment
does not indicate any collective attentional reception of these films. Thus, as the authors remark:
The finding that some films have low inter-subject correlation does not necessarily imply that the
viewers were not attentive to nor engaged/absorbed with the events in those films (ibid, p. 18),
which means an explicit admission that the experiment shows only an initial perceptive processing,
and that comprehension is a singular individual cognitive experience.
In the same article, they refer to a very interesting experiment about the dynamics of the
cognitive processing as a whole, and the effects of editing on this process (Hasson et al., 2008a).
They used clips of two Chaplin's silent films (The Adventurer, 1917 and City Lights, 1931), in its
original temporal structure and in three different temporal scaled editings, to demonstrate the
2

Cfr. Brown, 2011, for more arguments against Smith's claim.

4

dynamics involved in the cognitive processing of different orders of images. In the first temporal
scale, up to 4s, the fast processing of the visual stimuli was not disrupt or disturb by the different
editings, what confirms that the sensory regions are driven automatically by the stimuli,
independently of their editing. In the second temporal scale, up to 12s, which seems to be necessary
to integrate information across events, An example might be the processing of different kinds of
montage relationships between adjacent shots (Hasson, 2008b, p. 12), the inter-subject correlation
was affected. And in the third longer scale, up to 36s, that may be necessary to the processing of
the movie as a whole; for example, for inferring the characters motives, intentions, and beliefs, and
for processing the plot and predicting outcomes (ibid, p. 12), there was a clear dissociation
between the reliability of the responses and response amplitudes. As we have seen before, the lack
of inter-subject correlation does not mean the subjects are not attentively engaged on the cognitive
processing of the film. In fact, the authors remark large response amplitudes in the longer scale
regions for all the clips: We interpret the strong response amplitudes in these brain regions as
reflecting incessant processing, presumably aimed to extract meaningful information from the
stimuli (Hasson, 2008a, p. 2547). To conclude, it seems that at the single subject level, there was
an attentional effort to understand even the scrambled clips. Second, as the authors claim as well,
the different editing styles do have an effect on viewers cognitive processing. But, I remark, the
final comprehension of this subjective experience can not be fully determined by third person data.
Actually, I believe as well we need longer temporal scales to really comprehend the film, that is, to
integrate the inferences about the film with our higher internal mental states. As said before, this
higher integration can occur over very little time scales not detected by the fMRI.
Following Hasson's experiment, Dmochowski and colleagues realised a new one with the two
of the same films, The Good, the Bad and the Ugly (Leone, 1966), and Bang! Youre Dead
(Hitchcock, 1961). They measured the intra-subject correlation and the inter-subject correlation
using EEG down to one second resolution. This time, there are some indicators of emotionally
laden attention. But then, as the authors themselves recognise, these data indicate a stimulus-driven
processing, which does not imply an attentional processing of the stimuli. This could confirm my
hypothesis of a longer temporal scale for the subject's higher comprehension, as they did confirm
the temporal scaled processing claimed by Hasson's study. So, it seems that attention can arises but
not last enough for a higher order integration. Maybe it is due to the short duration of the clips,
maybe to the resolution and duration of the measurement with the EEG.
Comprehension is an individual conscious process, and even if third person data could
indicate some high order processing of information, first person access is essential for any analyses
of the subject's experience. The most interesting experiments are the event segmentation ones, as
5

they combine first person access and third person data (Zacks et al., 2007). The event model is the
basic structure for our comprehension, not only of films, literature, music, but also of real world
experiences. That is, we detect meaningful event units by perceiving the regularities and the
differences in a sequence of events. Therefore, we construct the cognitive model about the situation
presented, a cognitive model that plays a role in the memory about these events and in our
expectancies about what will come next. It is well recognised that event segmentation can reflects
reorientation of attention, but the arousal of attention needs a kind of violation in our expectations
(Corbetta et al., 2008).
Jeffrey Zacks is a reference on this field, but we cannot rely on his conclusions about film.
Based on experiments on event segmentation of Moonraker (Gilbert, 1979) and Le Ballon
Rouge (Lamorisse, 1956), Zacks and Magliano found that the participants individuated the
boundaries between events mainly in the films' scenes breaks (Magliano and Zacks, 2011; Zacks
and Magliano, 2011a); and that the subjects inferred correctly the intentions of Bond in the film
Moonraker (Zacks and Magliano, 2011a). So, they conclude that: Intentionally breaking the rules
of continuity editing may be artistically desirable and may not disrupt early perceptual processing of
a film. However, there may be costs for violating these conventions in terms of comprehension and
memory (Magliano and Zacks, 2011, p. 1511). As Zacks himself claims in another article:
Linking attention and particularly memory updating to prediction failures may be highly
adaptive. If prediction failures tend to occur at those points in time at which a meaningful new event
has begun, then that is just the time at which one should update one's mental models of 'what is
happening now.' [] This mechanism may be profoundly valuable for adaptively regulating
immediate behavior as well as for guiding long-term learning about one's environment. (Zacks et
al, 2011b, p. 4065). Actually, Zacks and Magliano do not present any evidence of an attentional
processing of the films, independently of their editing style.
Moreover, they use the film Le Ballon Rouge (Lamorisse, 1956), to argue for a superior
efficiency of the continuity rules of hollywood narratives to favour the event segmentation, and
therefore the comprehension of films. Well, the precise role Le Ballon Rouge played in film history
and theory, mainly because of Andr Bazin's article Le montage interdit, should be well-known:
Le Ballon Rouge is in fact an example of a film that does not follow the principle of the analytical
continuity editing, which is precisely the kind of montage that Bazin calls prohibited in the article.
Despite that, it seems that segmentation experiments are a very important source to analyse the
subjects comprehension, as well as to analyse social cognition.
Overall, these well-known experiments do not show any evidence of aesthetic attention 3. I
3

Thereareotherexperiments,withoutthepretenseofevaluatingthefims,thatgivesevidenceforanattentional
6

believe the matter is not only about the concept of aesthetics, but about principles and purposes in
general, as they reflex each other. I mean that effectiveness of inter-subject correlation, faster
reaction times, easy processing of information, response reliability, predicting efficiently the
outcomes, control of the viewers' mind, are not exactly what an aesthetic experience is about. Or,
to be more precise: what's the problem with the variability and the indetermination of the spectators
cognitive activity?

Bibliography:
Allport A. (2011). Attention and Integration. In Mole, Smithies and Wu (eds.) Attention:
philosophical and psychological essays, New York: Oxford University Press, pp. 24-59.
Brown W. (2011). Resisting the Psycho-Logic of Intensified Continuity. Projections, 5 (1), pp. 6986.
Corbetta M. et al. (2008). The Reorienting System of the Human Brain: From Environment to
Theory of Mind. Neuron, 58, pp. 306-324.
Dmochowski J. et al. (2012). Correlated components of ongoing EEG point to emotionally laden
attention a possible marker of engagement?. Frontiers in Human Neuroscience, 6, article
112, URL: http://journal.frontiersin.org/article/10.3389/fnhum.2012.00112/full
Genette G. (1999). The Aesthetic Relation. Ithaca: Cornell University Press.
Germeys F. and dYdewalle G. (2007). The psychology of film : perceiving beyond the cut.
Psychological Research, 71, pp. 458-466.
Hasson U. et al. (2008a). A Hierarchy of Temporal Receptive Windows in Human Cortex. Journal
of Neuroscience, 28 (10), pp. 2539-2550.
Hasson U. et al. (2008b). Neurocinematics : The Neuroscience of film. Projections, 2 (1), pp. 1-26.
Hochstein S. and Ahissar M. (2002). View from the Top: Hierarchies and Reverse Hierarchies in the
Visual System. Neuron, 36, pp. 791-804.
Hristova E. and Grinberg M. (2011). Time Course of Eye Movements during Painting Perception.
In Kokinov, Karmiloff-Smith and Nersessian (eds.) European Perspectives on Cognitive
Science,
Sophia:
New
Bulgarian
University
Press,
URL:
http://www.nbu.bg/cogs/eurocogsci2011/proceedings/pdfs/EuroCogSci-paper196.pdf
Kant I. (1790). Critique of the Power of Judgment. New York: Cambridge University Press, 2001.
Magliano J. and Zacks J. (2011). The Impact of Continuity Editing in Narrative Film on Event
Segmentation. Cognitive Science, 35 (8), pp. 1489-1517.
Mole C. (2009). Attention. In The Stanford Encyclopedia of Philosophy (Winter 2012 Edition),
URL: http://plato.stanford.edu/archives/win2012/entries/attention/
Schaeffer J.-M. (2013). Beyond Speculation: Art and Aesthetics without Myths. Calcutta: Seagull
processingofafilmandofactivityinthedefaultmodenetwork,asforexample: Pamilo et al. (2012). Functional
Subdivision of Group-ICA Results of fMRI Data Collected during Cinema Viewing. PLoS ONE, 7 (7), URL:
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0042000 .
7

Books.
Schaeffer J.-M. (2010). Why Fiction?. Lincoln: University of Nebraska Press.
Schaeffer J.-M. (2009). Thorie des signaux coteux, esthtique et art. Rimouski: University of
Quebec, Tangence diteur.
Searle J. (1983). Intentionality. An Essay In The Philosophy Of Mind. New York: Cambridge
University Press.
Smith T. (2012). The Attentional Theory of Cinematic Continuity. Projections: The Journal for
Movies and the Mind, 6 (1), pp. 1-27.
Zacks J. et al. (2007). Event Perception: A Mind/Brain Perspective. Psychological Bulletin, 133
(2), pp. 273-293.
Zacks J. and Magliano J. (2011a). Film, Narrative, and Cognitive Neuroscience. In Melcher and
Bacci (eds.) Art & the Senses, New York: Oxford University Press, pp. 435-454.
Zacks J. et al. (2011b). Prediction Error Associated with the Perceptual Segmentation of Naturalistic
Events. Journal of Cognitive Neuroscience 23 (12), pp. 4057-4066.