(Discourse Approaches to Politics, Society and Culture 55) Bertie Kaal (ed.), Isa Maks (ed.), Annemarie van Elfrinkhof (ed.)-From Text to Political Positions_ Text Analysis across Disciplines-John Ben (1).pdf

From Text to Political Positions
Discourse Approaches to Politics, Society

and Culture (DAPSAC)
The editors invite contributions that investigate political, social and cultural processes
from a linguistic/discourse-analytic point of view. The aim is to publish monographs
and edited volumes which combine language-based approaches with disciplines
concerned essentially with human interaction disciplines such as political science,
international relations, social psychology, social anthropology, sociology, economics,
and gender studies.
For an overview of all books published in this series, please see
http://benjamins.com/catalog/dapsac
General Editors
Ruth Wodak, Andreas Musolff and Johann Unger
Lancaster University / University of East Anglia / Lancaster University

r.wodak@lancaster.ac.uk; A.Musolff@uea.ac.uk and j.unger@lancaster.ac.uk
Advisory Board
Christine Anthonissen
Konrad Ehlich
Christina Schffner
Michael Billig
J.R. Martin
Louis de Saussure
Piotr Cap
Jacob L. Mey
Hailong Tian
Paul Chilton
Greg Myers
Teun A. van Dijk
John Richardson
Cardiff University
Barcelona
Luisa Martn Rojo
University of Portsmouth
Stellenbosch University
Loughborough University
University of d
Lancaster University
Universitat Pompeu Fabra,
Free University, Berlin

University of Sydney
University of Southern Denmark
Loughborough University
Aston University
University of Neuchtel
Tianjin Foreign Studies
University
Joanna Thornborrow
Sue Wright
Universidad Autonoma deMadrid
Volume 55
From Text to Political Positions. Text analysis across disciplines
Edited by Bertie Kaal, Isa Maks and Annemarie van Elfrinkhof

Text analysis across disciplines
Edited by
Bertie Kaal
Isa Maks
Annemarie van Elfrinkhof
VU University Amsterdam
John Benjamins Publishing Company

Amsterdam/Philadelphia
TM
The paper used in this publication meets the minimum requirements of

theAmerican National Standard for Information Sciences Permanence
of Paper for Printed Library Materials, ansi z39.48-1984.
Library of Congress Cataloging-in-Publication Data

From Text to Political Positions : Text analysis across disciplines / Edited by Bertie Kaal,
Isa Maks and Annemarie van Elfrinkhof.
p. cm. (Discourse Approaches to Politics, Society and Culture, issn 1569-9463 ; v. 55)
Includes bibliographical references and index.
1. Discourse analysis--Political aspects. 2. Public communication--Political aspects.
3.Mass media--Political aspects. 4. Communication in politics. I. Kaal, Bertie.
P302.77.F76
2014
401.41--dc23
2014004430
isbn 978 90 272 0646 6 (Hb ; alk. paper)
isbn 978 90 272 7034 4 (Eb)
2014 John Benjamins B.V.

No part of this book may be reproduced in any form, by print, photoprint, microfilm, or any
other means, without written permission from the publisher.
John Benjamins Publishing Co. P.O. Box 36224 1020 me Amsterdam The Netherlands
John Benjamins North America P.O. Box 27519 Philadelphia pa 19118-0519 usa
Table of contents
Foreword
chapter 1
Positions of parties and political cleavages between parties in texts
Jan Kleinnijenhuis and Wouter van Atteveldt
vii
1
Part I. Computational methods forpoliticaltextanalysis

Introduction23
Piek Vossen
chapter 2
Comparing the position of Canadian politicalparties using French
and English manifestos as textual data27
Benot Collette and Franois Ptry
chapter 3
Leveraging textual sentiment analysis withsocial network modeling:
Sentiment analysis of political blogs in the 2008 U.S. presidential election
Wojciech Gryc and Karo Moilanen
chapter 4
Issue framing and language use intheSwedish blogosphere:
Changing notions of the outsider concept
Stefan Dahlberg and Magnus Sahlgren
chapter 5
Text to ideology or text to party status?
Graeme Hirst, Yaroslav Riabinin, Jory Graham,
Magali Boizot-Roche, and Colin Morris
chapter 6
Sentiment analysis inparliamentaryproceedings
Steven Grijzenhout, Maarten Marx, and Valentin Jijkoun
47
71
93
117
chapter 7
The qualitative analysis of political documents135
Jared J. Wesley
vi
Part II. From text to political positions viadiscourseanalysis

Introduction163
Veronika Koller
chapter 8
The potential of narrative strategies inthediscursive construction
of hegemonic positions and social change171
Nicolina Montesano Montessori
chapter 9
Christians, feminists, liberals, socialists, workers and employers:
The emergence of an unusual discourse coalition
Anja Eleveld
chapter 10
Between the Union and a United Ireland: Shifting positions
in Northern Irelands post-Agreement political discourse
Laura Filardo-Llamas
chapter 11
Systematic stylistic analysis: The use of a linguistic checklist
Maarten van Leeuwen
chapter 12
Participation and recontextualisation innewmedia:
Political discourse analysis and YouTube
Michael S. Boyd
189
207
225
245
Part III. Converging methods

Introduction271
Alan Cienki
chapter 13
From text to the construction ofpoliticalparty landscapes:
A hybrid methodology developed forVotingAdviceApplications
Andr Krouwel and Matthew Wall
chapter 14
From text to political positions: The convergence
of political, linguistic anddiscourseanalysis
Annemarie van Elfrinkhof, Isa Maks, and Bertie Kaal
275
297
About the authors
325
Index
331
Foreword
There is clearly a need for the identification and positioning of political parties and
their stances. This need is both of a societal and of an academic nature. Socially
it seems that traditional voting patterns have dissolved and information sources
have diversified exponentially. Voters no longer vote primarily according to their
community identity, such as class or religion (Franklin et al. 1992), but they make
their choice independently. Accessible information about party positions on the
social structure in general as well as on specific issues is therefore imperative to
inform the public so that they can cast an informed vote.
Academically, identifying policy preferences and positions of political parties
and their actors is primarily a concern for political scientists. When analysing
the behaviour of political parties over time or across countries (political) scientists need reliable and valid measures to establish positions. The search for such
estimates has resulted in a range of analytical methods ranging from expert surveys, to opinion polls and roll-call behaviour to various forms of content analysis
(Krouwel and Van Elfrinkhof 2013). Each of these methods has its advantages and
disadvantages, resulting in more or less reliable, valid and transparent measurements that might be improved by adopting cross-disciplinary research designs.
This volume contributes to the literature of party positioning by focusing on
political text analyses with a wide variety of approaches from political science,
linguistics and discourse analysis. The central question is how to identify verbal
expressions of politically motivated ideas in texts. Political texts in particular function to articulate ideas persuasively to a broad audience. For this reason, political
texts reflect the goals and policies induced by the norms and values of a partys
ideology and identity. As such, texts are an excellent source of political positions
and their rationale. Furthermore, the sheer abundance and availability of political
texts make them an attractive source for analysis, especially since collections of
these texts are often available digitally. The chapters in this volume discuss methods to analyse a variety of political text genres that are designed to establish and
communicate party positions, including newspaper articles, election manifestos,
campaign and parliamentary speeches, online documents, blogs and interviews
(see Table 1).
The purpose of this volume is to discover how stance is encoded in political
texts and how such characteristics can best be gauged on political dimensions.
viii From Text to Political Positions
Simply reading and interpreting a text will not do, as meaning making and the
social implications of policies are subject to their dynamic and diverse contexts,
as analysed in qualitative studies. Another approach is to analyse large amounts
of political text to find variations in content, linguistic and discursive aspects that
can be linked to positions on political dimensions. Current types of analysis apply
qualitative and quantitative methods, or combinations thereof. Such attempts at
modelling political text analysis for party positioning reveal the complexity of
sense making. Both qualitative and quantitative approaches are pursued in many
of the contributions in this volume.
We have tried to divide the variety of approaches into meaningful parts that
give an impression of the division between qualitative and quantitative methods,
as well as attempts to converge them with reflections on the flaws and advantages of mixing methods. As an introduction to the problematics addressed,
Kleinnijenhuis and Van Atteveldt (Chapter 1) describe the evolution of political
text analysis from the early concept of political thought (Machiavelli) to modern
advances in the domains of knowledge representation, natural language parsing,
automated content analysis and semantic-web technologies. That chapter then
focuses on conceptualisation through the logic and structure of semantic networks and discusses the sophisticated NET-Method, developed at The Network
Institute (VU University Amsterdam). Part I, introduced by Piek Vossen, is a
collection of discussions about automated, quantitative methods to identify
characteristics and variation in political texts. Part II, introduced by Veronika
Koller, focuses on qualitative methods, some of which apply quantitative analysis
as evidence of the existence of linguistic and discursive aspects of meaning making. Part III is introduced by Alan Cienki and includes two chapters on projects
attempting to mix and match qualitative and quantitative methods for better
results and best practice. The purpose of this book is to present the variety of considerations and applications of text analysis for political party positioning, and to
inform scholars of the wide range of available methods that have their advantages
and disadvantages under different conditions. More ambitiously, this book aims
to trigger cross-disciplinary research in which methods are combined, as a way
to refine research methods for the analysis of stance taking and to achieve more
accurate results overall.
However, attempts to converge quantitative and qualitative methods of content analysis run across the three parts, as shown in Table 1. For instance, the
chapters by Kleinnijenhuis and Van Atteveldt, Collette and Ptry, and Krouwel
and Wall show the ability and the promise of quantitative methods for the analysis
of large amounts of texts vis--vis the in-depth analysis of qualitative methods.
In linguistics, various quantitative methods have been developed for analysing
Foreword
ix
expressions of subjectivity and sentiment in party documents as well as their rhetorical affordances. The chapters by Gryc and Moilanen, Dahlberg and Sahlgren,
and Grijzenhout, Marx and Jijkoun show how adding knowledge of linguistic features can enhance the potential of quantitative methods. The focus of discourse
analysis is on extracting constructions of meaning and analysing the persuasive
nature of political discourse in its social context. Montesano Montessori, Eleveld,
Filardo-Llamas, and Van Leeuwen show how linguistic formulations and conceptualisations relate to the social context.
Table 1. Qualitative and quantitative methods and genres across chapters.
Ch. Authors
Data source
Methodology
Quant/Qual
Kleinnijenhuis &
Van Atteveldt
Newspapers
NET Method
Quantitative
Collette & Ptry
Election manifestos Wordfish, Wordscores Quantitative
Gryc & Moilanen
Political blogs
Entity-centric
Quantitative
document-level
sentiment classification
Dahlberg & Sahlgren
Blogs, websites
political parties
Random Indexing
Quantitative
and Qualitative
Hirst, Riabinin, Graham, Parliamentary

Boizot-Roche & Morris proceedings
Machine Learning
Quantitative
Grijzenhout, Marx
& Jijkoun
Parliamentary
proceedings
Sentiment analysis
Machine Learning
Quantitative
Wesley
Political texts
Qualitative document
analysis
Qualitative
Montesano Montessori
Speeches and
declarations
Critical Discourse/
narrative analysis
Qualitative
Eleveld
Interviews and
other documents
Discourse/narrative
analysis
Qualitative
10 Filardo-Llamas
Speeches
Critical discourse
analysis
Qualitative
11 Van Leeuwen
Parliamentary
speeches
Stylistic analysis
Qualitative
12 Boyd
Text comments on
political videos
Critical discourse
analysis
Quantitative
and Qualitative
13 Krouwel & Wall
Election manifestos, Content analysis

party documents
Quantitative
and Qualitative
14 Van Elfrinkhof, Maks

& Kaal
Election manifestos Wordscores,

subjectivity analysis,
Critical Discourse
Analysis
Quantitative
and Qualitative
Whether assessing political texts quantitatively or qualitatively, researchers

are confronted with strategic and creative language use: politicians are creative
language users, and political language as a result changes fast (Grijzenhout et
al., page 130) and these functional aspects also deserve attention. Filardo Llamas
stresses the ambiguity and vagueness of political language which allow for multiple interpretations of the same texts that can divide or unify parties, as was the
case with the peace settlement in Northern Ireland. The possibility of multiple
interpretations is also the topic of Elevelds chapter that discusses how one common policy narrative can be supported by different political parties and lead to
unusual coalitions. Hirst et al. discuss unexpected and hidden structures, where
polarization is not an ideological opposition, but rather a pragmatic attack and
defense dichotomy. These examples of political text analyses give evidence of the
researchers challenges and opportunities.
With a view to make the collaboration between disciplines tangible, we
organised the workshop From Text to Political Positions at the VU University
Amsterdam, in April 2010. This led to discussions on work in progress, on the
challenges of the complexity of the material to be analysed for relations between
language and politics, and on finding solutions to bringing qualitative and quantitative methods together. This volume includes a selection of the projects presented
at the workshop that have since been revised and updated. The result is a representative variety of approaches to political text analysis across genres, languages,
theories and methods. It brings together state-of-the-art political, discourse and
linguistic analytical models that make links between linguistics and the social
sciences. The chapters present top-down and bottom-up methods and applications using a variety of texts, each with their own communicative functions. Each
chapter reflects on advantages and disadvantages of the methods that were chosen
in view of their relation to the data and the reliability of their results.
Developments in the field of political text analysis have been recorded in this
volume with the aim to take notice of the challenges of cross-disciplinary analytical models as well as the complexity of the relationship between content, language
use and different motivations for political action.
Amsterdam, November 2013

Bertie Kaal
Isa Maks
Annemarie van Elfrinkhof
Foreword
xi
Reference
Franklin, M., T. Mackie, and H. Valen. 1992. Electoral Change: Responses to Evolving Social and
Attitudinal Structures in Western Countries. Cambridge: Cambridge University Press.
Krouwel, A. and A. van Elfrinkhof. 2013. Combining strengths of methods of party positioning
to counter their weaknesses: the development of a new methodology to calibrate parties on
issues and ideological dimensions. Quality & Quantity, DOI: 10.1007/sl 1135-013-9846-0.
Acknowledgements
The research project From Text to Political Positions and the workshop from which these chapters resulted was made possible by the VU institute for interdisciplinary research, The Network
Institute (formerly the Center for Advanced Media Research Amsterdam, CAMeRA). Other
sponsors of the workshop were Kieskompas and the Nederlandse Nieuwsmonitor. We would
like to thank all participants for their presentations and lively discussions. We are grateful to the
contributors to this volume for their patience and open-mindedness to make revisions and to
follow up on the editors and reviewers suggestions. Special thanks are due to Alan Cienki for
his critical comments and meticulous editing.
chapter 1
Positions of parties and political cleavages

between parties in texts
VU University Amsterdam, Department of Communication Science
If humans are political animals, and language is their most versatile communication tool, then the old question about what should be extracted from political
texts to understand politics deserves on-going attention. Recent advances in
the information and communication sciences have resulted in new means to
process political texts, especially advances in the domains of knowledge representation, natural language parsing, automated content analysis and semantic
web technologies. However, applying these innovations to uncover what matters
in politics is far from trivial. The aim of this chapter is to give an introduction to
an analysis of political texts aimed at inferring their political meaning.
First, a few concepts from the history of political thought will be reviewed so as to
arrive at a feeling for what should be revealed by means of an analysis of political
texts. Next, three sets of Natural Language Processing tools will be distinguished
to analyze political texts: tools to assess whether a concept (or object) occurs,
tools to assess whether an (asymmetric) relationship between concepts occurs, and
tools to assess the nature of the relationship between concepts. These three sets of
tools relate to advances in ontology construction and entity recognition, advances
in statistical associations and network theory, and advances in part-of-speech tagging and grammar parsing respectively. The aim is to show how automation of the
three sets of tools could be employed in the near future and could give reliable and
valid answers to frequently asked questions in political communication.
Political language and content analysis

Websters Unabridged Dictionary defines politics in various ways. Politics is the
total complex of interacting and usually conflicting relations between men living
in society, it is concerned with governing or with influencing or winning and
holding control and with actions, practices or policies to achieve goals with
respect to issues. In short, politics is usually about achieving goals by means of

policies, about conflict or cooperation, and about winning or losing.
Machiavelli (14691527) observed that princes can acquire new principalities by means of their own virtue, or by fortune. Their virtue prescribes them to
take great pains to satisfy the people and make them content with laws that serve
their issue interests, but also to raise prestige by the art of conflict and cooperation, thus by revealing themselves without any preservation in favor of one side
against another, with the risk of vigorous wars. Fortune is what befalls on leaders,
such as unexpected natural disasters, or popular support. Virtue can be practiced
whereas fortune can only be anticipated. Machiavellis primary concern is political
language rather than politics itself, that is, how others would speak about the acts
of actors and about what befalls upon them. Even winning a war was perceived
by Machiavelli as the art of inspiring the army with confidence in itself and in its
general, thus as the art of inducing others to speak about themselves in relationship to their leader. Interpreted from the contemporary perspective of this chapter,
already Machiavelli elucidated that political language centers around four types
of statements:1
Virtue: what an actor can do:
1. take an issue position (pro or con a cause, e.g., poverty, crime, unemployment);
2. cooperation or conflict with support or criticism from other actors (e.g., building a coalition government, waging war)
Fortuna: what befalls upon an actor:
3. real world developments with regard to issues (e.g., famine, unemployment)
4. actors success or failure, gains or losses (e.g., gains and losses in wars, opinion
polls, or in political debates).
Machiavelli is considered as the forerunner of the age of absolutism, but his basic
ideas are equally important for a democracy. In a democracy, each of these four
types of statements is important to attract voters. Voters choose a party when
they agree with the issue positions of that party (Tomz and Van Houweling 2008;
Westholm 1997), and even when the issues on which that party holds a strong
reputation dominate the campaign (Budge and Farlie 1983; Hayes 2005; Petrocik
1996). Voters choose a party with a strong profile in terms of attacks and criticisms from political adversaries and support from within and from societal actors
(Shah et al. 2002). They prefer incumbent parties in case of favorable real world
developments (e.g., economic growth) and a challenger in case of deteriorating
1. This interpretation of Machiavelli is based especially on Chapters 67, 1718, 21 and 25 of
The Prince and Chapters 3335 of The Discourses.
Chapter 1. Positions of parties and political cleavages between parties in texts
real world developments (e.g., unemployment) (Schumpeter 1950 (1943)), but in

the absence of objective knowledge they are susceptible to the positive or negative
portrayal of these developments in the media (Hetherington 1996; Sanders and
Gavin 2004; Soroka 2006). Of these four driving factors, the attribution of successes and failures to parties in the media is the most important predictor of shifts
in party preferences (Kleinnijenhuis et al. 2007).
Rather unsuccessful attempts have been made to proclaim one of these four
types of statements as the most fundamental one. In their seminal study of the historical origins of party systems and voter alignments Lipset and Rokkan observed
that parties as well as voters loyalty to these parties rest on old conflicts and cleavages between parties (Lipset and Rokkan 1967), for example between workers and
owners (between the rich and the poor, the haves and the have-nots), or between
permissive, secular, urban, orthodox religious and rural groups (e.g., in the Dutch
context the rekkelijken versus preciezen in the sixteenth and seventeenth century). From the perspective of Lipset and Rokkan, disagreement about issue
positions mirrors the historical dividing lines of conflicts or cleavages between
actors. A central claim in Marxist theories is that that political-issue positions
simply mirror class cleavages, i.e., one particular type of cleavages. An opposed
view is apparent from the literature about issue voting and election campaigns.
Disagreement about issues tends to be seen as the heart of politics. Attack politics and news about conflicts between parties are either regarded as a mirror of
issue (dis)agreement, or as degenerated forms of political communication that
merely enhance political cynicism (Ansolabehere and Iyengar 1995; Cappella and
Jamieson 1997; Patterson 1993). The intellectual origins of this opposed view date
back to ideas about political apathy of Alexis de Tocqueville (De Tocqueville 1951
[1835]). Jon Elster (2009) shows that De Tocquevilles thoughts can be rewritten in the language of current social-science methodology as reciprocal causal
assertions, as two-way interactions, between developments at the societal level
(e.g., issue positions of parties, press coverage) and developments at the level of
citizens (beliefs, desires, preferences, behaviors). De Tocquevilles ideas are also
at the heart of Elisabeth Noelle-Neumanns (1980) theory of the spiral of silence,
which maintains that dominant issue positions in the media will have as an effect
that citizens with opposing views will feel less free to express their opinions, which
will in turn reinforce these dominant issue positions. The comparative research
literature on party manifestos (Budge, Robertson, and Hearl 1987; cf. the chapters by Collette and Ptry; Van Elfrinkhof, Maks and Kaal; and Krouwel and Wall
in this volume) also contributes to the view that issue positions drive cooperation and conflict between parties. As Van Elfrinkhof, Maks and Kaal point out
(this volume), a striking feature of the genre of party manifestos is that they deal
solely with the issue positions of a single party, rather than, for example, with the
conflicts and the teamwork within that party that produced a particular manifesto,
or with the pattern of conflicts and cooperation that will emerge in election campaigns, in government coalition negotiations, or within the next government coalition. Remarkably little longitudinal empirical research has been done, however,
to verify whether one fundamental type of statement is systematically mirrored
in other types of statements.
The simple truth may be, that a single causal order is impossible or unlikely.
Disagreement about a topic is sometimes the result of conflicts between actors,
but conflicts between actors may also follow from disagreement about a topic.
Sometimes the two may be unrelated, for example when a political party attacks an
ideological similar party, or induces third parties to attack or to neglect an ideological similar party, because precisely ideological similar parties are serious electoral
competitors. Winning or losing may be the outcome of conflicts, but vesting a
reputation as a loser may also cause conflicts, as Machiavelli stated over and over
again. Longitudinal research is needed to answer the important research question
as to which causal order is likely under which conditions. However, to arrive at rele
vant data for this type of research new tools are required to analyze the emphasis,
turns, shifts and moves in political language, both comprehensively and in-depth.
Meta-language about political language

More than two thousand years ago the ancient Greeks invented political theatre,
political dialogues and democracy. We still use their concepts to talk about political language, such as sign and signifier, symbols, and last but not least the subject-object-predicate triplet. Although the Greek concepts of subject, object and
predicate are ambiguous and outdated from the point of view of todays theories
about grammar, logic and semantics, they are still valid to discuss what language
is all about.
The Australian linguist Robert M.W. Dixon observes that in all languages
sentences deal with a subject: who or what directs its action or energy towards
a target or object (Dixon 1992, 2005). The nature of this action, or energy, is a
two-place predicate. The subject and the object are either animate entities, which
we will label actors, or circumstances. Other non-animate entities, will be labeled
as issues here, although in non-political context labels such as variables, circumstances or states of affairs would presumably be more intuitive. Subject-predicateobject triples resemble the a-symmetric xRy-triples in relational logic, which was
pointed out succinctly by Ludwig Wittgenstein (19891951) in his famous statement: Namen gleichen Punkten, Stze Pfeilen, sie haben Sinn (names resemble
points; propositions resemble arrows, they have sense [Wittgenstein 1922:3.144]).
The boundaries of ones propositions would also be the boundaries of ones world,
according to the early Wittgenstein, but in his later work he was precisely interested in the exchange and the misunderstandings between different world views
(Wittgenstein 1953), thereby recognizing the source of propositions as an integral
part of the proposition itself (effectively s: xRy, instead of merely xRy, in which s
represents the source, x the subject, R the predicate, and y the object).
Taking the analysis of propositions one step further, Fritz Heider (18961988),
another Austrian who moved into the Anglo-Saxon world, developed balance
theory in a remarkably short paper. Balance theory also deals with triangles of
three statements and assumes that the third relationship can usually be predicted
correctly from the first two on the basis of the principle that friends of friends, but
also enemies of enemies, tend to be friends, whereas enemies of friends, as well
as friends of enemies, will usually be enemies. Thus, if x dislikes y whereas y likes
z, then the expectation is raised that x dislikes z as well. In Heiders notation, in
which ~L means the opposite of the liking-relation L this would be: x~Ly and yLz,
therefore x~Lz. Balance theory, and later theories of cognitive consistency, such as
congruence theory and the theory of cognitive dissonance, hold that people will
try to avoid cognitive representations that violate balance by a number of Freudian
escape routes, such as the negation of information, blaming the messenger, or the
rationalization of previous choices with ingenious new arguments (Severin and
Tankard 2005).
Charles Osgood was the first to develop a coding instruction to extract xRystatements from full sentences and complete texts (Osgood, Saporta, and Nunally
1956). Many different elaborations of this method have been proposed (Deetjen
1977; Kleinnijenhuis 2008; van Cuilenburg, Kleinnijenhuis and De Ridder 1986).
Since xRy-statements build up a network, and so the enterprise to extract them
from texts has been labeled as semantic network analysis (Krippendorff 2004; Van
Atteveldt 2008), which is the topic of the next paragraph.
Elementary xRy-statements may look all too familiar from the point of view
of contemporary logic as applied in semantic web approaches (Antoniou and Van
Harmelen 2004), but it should be pointed out that in logic the focus is usually
on deriving theorems from sets of axioms in which the predicate R is invariant
between axioms, or at least predicates R, S, T, differ categorically from each
other. However, the predicates in political language, and in theories of balance
and cognitive consistency can be mapped onto a positive-negative continuum. In
contemporary logic, axioms that give rise to contradictions are deemed untenable and therefore uninteresting, whereas in cognitive consistency theories the
primary diagnostic of belief systems is the degree to which they are unbalanced
and therefore ambiguous when it comes to drawing inferences. In most logics,
statements are either true or false, whereas in political language statements have
both a magnitude (a frequency, a saliency), and an angle (a direction, a continuous positive-negative scale): they mount up to a vector model (Kleinnijenhuis and
Pennings 2001). Moreover, the direction of a predicate has not only a mean interpretation as its first statistical moment, but also higher-order moments, such as
variance, skewness and kurtosis. In short, contemporary logic aborts where political language, political dialogues and cognitive consistency theories start, namely
after a contradiction and after a variety of interpretations.
Three tools for analysing political texts

Although the aim of this chapter is to address whether political meanings can be
inferred from texts by automated extraction of subject-predicate-object-triples
from texts, we will start out with two more basic questions. Is it feasible to automate
a textual analysis to extract the occurrence and co-occurrence of concepts in texts?
From a theoretical perspective this question is equivalent to the question whether
it is possible to automate first-order and second-order agenda-setting research.
Does a concept occur? First-order agenda setting and entity recognition

Agenda setting (McCombs 2004; Rogers, Dearing and Bregman 1993), that is to
say first-order agenda-setting, is concerned with the transfer of issue saliency of
the agenda of one actor to the agenda of another actor. Agenda setting theory
predicts, for example, that significant media attention to a particular issue will be
followed by huge public attention for that issue. A transfer of issue saliency is also
at the heart of the issue ownership theory which attempts to explain party competition. The issues that are prominent in party manifestos will tend to become
the issues that are central in election campaigns. Pre-established reputations of
parties with respect to the issues that dominate the campaign, that may well go
back to the old cleavages between parties, determine which party will win at the
elections, and ultimately also determine the policy emphasis and policy expenditures of parties in government. In short, party competition is primarily directed
at increasing the emphasis on owned issues, rather than at pro- or contra arguments (Budge and Farlie 1983; Budge et al. 2001; Hayes 2005; Petrocik 1996). In
mediatized democracies, politics is basically the politics of attention (Jones and
Baumgartner 2005).
If the emphasis on issues and attributes could be measured with single-word
lists or with elementary boolean search strings, then it would be easy to automate
agenda setting research, but this is often not the case. However, ontology-matching may nevertheless help to automate agenda-setting research.
The ontological problem: (named) entity recognition

Let us start with an example article to elucidate the ontology-matching problem
that will be encountered when one tries to count the occurrence of concepts,
rather than the occurrence of words. The example article was compiled from a
number of available articles about the Israeli-Palestinian conflict. Scholars in the
tradition of Critical Discourse Analysis would regard texts from Western newspapers about the Israeli-Palestinian conflict as examples of us-them thinking.
For example, in Richardson (2004) a positive self-representation of the Western
world as a civilizing force is combined with a negative other-representation of the
Muslim world, including military threat, extremism and terrorism, despotism and
sexism. However, the bottom-up analysis of the sample article that is presented
below does not aim at such a high-level interpretation.
1. Time running out for Mideast two-state solution.
2. Only a few months ago, president Obama welcomed Israeli Prime Minister
Benjamin Netanyahu and Palestinian Authority President Mahmoud Abbas
to the White House.
3. President Obama said both leaders came to Washington in an effort to restart
the peace process and reach the goal of a two-state solution that ensures the
rights and security of both Israelis and Palestinians.
4. Obama told reporters that the Israeli government and the Palestinian
Authority had taken important steps to build mutual confidence since May.
5. Since then, president Abbas stated repeatedly that only a complete Israeli settlement freeze would create the conditions for a return to the negotiating table.
6. Today not only Israel, but also the United States dropped the Palestinian
demand for a settlement freeze that would have opened up negotiations
shortly.
7. If the past two years have shown nothing else, it is that the weak Ramallah
government will not realize enough success to help lead the path back to negotiations that bring about a two-state solution.
8. No two-state solution is possible without Hamas, but Israel and the United
States do not want to negotiate with Hamas.
9. Hamas leader Khaled Mashal called for continuing the jihad against Israel and
categorically denied any possibility of talks with Israel.
Readers who understand this article, must have an ontology in mind of existing
concepts and their relations. Ontology is the study of the things that are, and an
ontology is a name used in Knowledge Engineering to denote a (shared) formalization of a view on the world. Table 1 presents a simple strictly hierarchical ontology (or taxonomy) to match the content of the example article.
Table 1. An ontology to match the content of the sample article.

Actors
Palestinians

government: Palestinan Authority

leader: President Mahmoud Abbas

capital: Rahmallah

United States

leader: President Barack Obama
Hamas

leader: Khaled Mashal
Israel

leader: Prime Minister Benjamin Netanyahu
Issues

peace process, Israeli-Palestinian negotiations, peace talks

two-state solution

Israeli settlements

settlement freeze []
An easy observation is that pre-established ontologies are indispensable, since

otherwise the article would become incoherent (what has Ramallah to do with
Abbas?). Explicit ontologies enable automation of agenda setting at the level of
concepts, and not just only at the level of words or boolean search strings of words.
From an automation perspective the advantage of an ontology is its additivity.
Frequency counts of lower-order concepts (e.g. Abbas, Mashal) are sufficient
to arrive at counts of higher-order concepts such as the Palestinians or Hamas.
Antonyms and concepts with an opposed meaning give rise to a complication.
Lower order antonyms of higher order concepts are marked with a []-mark. The
[]-mark in the example ontology means that attention for a settlement freeze can
be counted as attention for Israeli settlements, but also that protagonists of a settlement freeze should be counted as antagonists of Israeli settlements, and vice versa.
Research on election campaigns or government policy ontologies typically
deals with 500 up to 2500 lower-order concepts. A lower number of concepts is
often desirable when it comes to interpretation and dispatches. To achieve this, the
concepts may be mapped in accordance with the rules implied in the ontology to
5 up to 25 relevant concepts.
Co-occurrence of concepts? Conditional probabilities and associative framing

Second-order agenda setting maintains that the attributes that are associated with
an object in the media to which one is exposed will also become the attributes
that the audience will associate with the concept (McCombs 2004). Second-order
agenda setting rests on the transfer of saliency, as did first-order agenda setting.
The question that needs to be answered next is what causes that a concept B
(dotted line) is attributed to, or associated with a concept A (solid line), as shown
in Figure 1? In his seminal article Features of Similarity, Amos Tversky points out
that the smaller set often lets you think of the larger set simply because the number
of elements in the intersection of two sets is a larger percentage of the number
of elements in the smaller set than of the number of elements in the larger set
(Tversky 1977).
P (solid dotted) = 2/5 = 0.4

P (dotted solid) = 2/10 = 0.2
Figure 1. Conditional probabilities and asymmetric associative framing.
For example, if you think of kerosine, you may come to think about an airplane,
but it is quite unlikely that thinking about an airplane will immediately generate
thoughts about kerosine. Planes are more strongly associated with visions of holiday or congress destinations. In the study of language, the size of sets is equivalent
to the occurrence of concepts, whereas their intersection is equivalent to their
co-occurrence.
After 9/11 politically correct journalists attempted to show that most Muslims
(a large set) were actually not terrorists (a small set) by giving examples of hardworking integrated Muslim immigrants, but this did not prevent thoughts about
terrorism from generating pictures of 9/11 and negative ideas about Muslims.
Conditional probabilities give the cue to associative framing of topics in the media,
for example, of terrorism and Islam (Ruigrok and Van Atteveldt 2007). In Spain,
however, even the Madrid train bombings by Muslim terrorists shortly before the
national elections in 2004 were exploited by the governing Partido Popular (PP)
as new evidence of ETA violence. In spite of the Madrid bombings, in spite of the
protests against the PP propaganda, and in spite of the victory of Spain after almost
a millennium of servitude to Muslim invaders, the Spanish press never came to
associate terrorism strongly with the Islam in the period between 20002008. The
primary association of terrorism remained with ETA. Primary associations in the
press of immigrants were not the Islam, or terrorism, but rather the economy
and the calling effect of the regularization of immigration (Mena-Montes 2010).
Associative framing can be automated fully, since techniques to count occurrences
10
and co-occurrences of words are straightforward, whereas progress in ontology

matching also enables counting the occurrence and co-occurrence of higher-order
concepts.
Many scholars have been puzzled by conditional probabilities, for example,
John Maynard Keynes who wrote in 1929, after his renewed encounter with Ludwig
Wittgenstein in Cambridge: Well, God has arrived. I met him on the 5:15 train,
thereby referring to proposition 5.15 (Wittgenstein 1922), one of Wittgensteins
propositions about conditional probability in terms of sets. Conditional probabilities are also at the heart of Bayesian statistics. Many theorems from Bayesian
statistics have an analogue in associative framing.
When the frequency of concept occurrence per textual unit is known, it is
simple to compute the asymmetrical associations or co-occurrence between concepts. These associations can be conceptualized as the conditional probability of
encountering one concept, given that another object is encountered: given that
a sentence contains a reference to Hamas, how likely are we to see a reference to
Israel? This conditional probability is the association between Hamas and Israel.
Taking the sentence as the contextual unit, in the sample text this probability is
100% as both sentences mentioning Hamas also mention Israel. However, the
reverse is not true, as only 2 out of 6 sentences mentioning Israel also mention
Hamas, making the association between Israel and Hamas 33%. Figure 2 shows the
network of all associations greater than 50% as extracted from the example text.
The lower right shows a central cluster of strongly interconnected actors: Israel,
USA, and Palestinians. Interestingly, Palestinians and Hamas are not associated
with each other at all. Moreover, while both are associated with the Peace Process
and the Two-state Solution, these issues are not associated with Hamas and the
Palestinians, but rather with Israel and the USA. If this article would be representative of Middle-East reporting, one would expect that people think of the USA and
Israel when they think of the peace process, and not of Hamas or the Palestinians.
Peace
process
Hamas
Two-state
solution
Israel
USA
Figure 2. Association network extracted from the sample text.
Palestinians
Semantic network analysis

In the examples above we have shown how the occurrence and co-occurrence
of words can provide meaningful information on the agenda and associations of
relevant concepts. These techniques do not tell us, however, how these concepts
are related. Fully understanding the relations expressed in language requires fully
understanding both the intricacies of natural language and the context in which
the language is to be understood. This is beyond the capabilities of the computer.
However, it is possible to employ grammatical analysis to analyze some of these
relations. First, we should understand how humans would analyze the example
text on the two-state solution.
Manual coding using the NET-method

Presumably, the most straightforward way to understand the extraction of political
statements from political texts is to present the extracted statements in a network.
Figure 3 represents the statements about the two-state solution from the sample
text based on human coding that are attributed to president Abbas and to president Obama, whereas Figure 4 represents the network according to the compiler
of the article. In Figure 3 and 4 lower-order concepts (e.g., Obama, Abbas) are
mapped in accordance with the tree-structure of the ontology to higher order concepts (e.g., USA, Palestinians). Solid arrows represent positive associations; dashed
arrows represent negative associations. The arrow labels include the quoted actor,
an abbreviation for the type of statement, the sentence number in the example
article on which the arrow is based, and a few crucial words from the predicate
which clarify why these relationships are positive or negative. Table 2 gives an
overview of the abbreviations used.
Two additional statement types that were not discussed before, pop up in the
example article: CSQ (= consequences of issues for actors) and CAU (= causal
relationships between issues).
Figure 3 shows that, according to Obama, Israel, the Palestinians and the
peace process are positively associated with each other in each direction. They
mutually trust each other, both can benefit from the rights and security delivered
by the peace process, and both want to restart the peace process. According to
Abbas, however, a settlement freeze is a precondition for the peace process.
11
12
Israel
Settlement freeze
USA:
CSQ 3:
ensures rights
and security
USA:
CC 4:
mutual
conflidence
USA:
CC 4:
mutual
conflidence
USA:
IP 3:
restart
Palestinians:
CAU 5:
create conditions
Peace process
USA:
CSQ 3:
ensures rights
and security
USA:
IP 3:
restart
Palestinians
Figure 3. Quotations from Obama and Abbas, attributed respectively to the USA
and the Palestinians.
CC 8:
not
negotiate
USA
CC 2:
welcomed
CC 6:
drops CC 2:
welcomed
CC 8:
not
negotiate
CC 9:
jihad
Israel
CC 6:
drops
Palestinians
CC 9:
denied
possibility
SF 7: weak &
not success
IP 7:
not
lead back
IP 6:
demand
Settlement
freeze
Hamas
CAU 5:
open up
Peace
process
CAU 7:
bring
IP 8:
without
support
Reality
REA 1 & 8:
time running out
& not possible
Two-state
solution
Figure 4. Statements from the sample news article on behalf of its author.
Table 2. Statement types and their abbreviations.
2-place
predicates
1-place
predicates
Abbreviations of
statement type
Subject/
agent
Object/
target
Example
IP: Issue Position
actor
issue
Obama freezes bonuses
CC: conflict/cooperation,
Support/Criticism
CSQ: consequences
actor
actor
issue
actor
Palin unleashes attack

against Obama
Bonuses are simply good for
bankers
Bonuses help the economy
CAU: Causation
issue
issue
REA: Real World

developments
SF: Success/failure
reality
issue
reality
actor
Bonuses rose further in

2011
Obama has lost heavily
AEV: Actor Evaluations
actor
ideal
Obama is doing a great job
IEV: Issue Evaluations
issue
ideal
Bonuses are obscene
In contrast with Figure 3, negative relationships show up in Figure 4. Actually,

only a few relationships are positive. The USA welcomed the Palestinians, who
demanded a settlement freeze, which could open up the peace process, which
could bring about a two-state solution. In line with the transitivity principle,
this chain of reasoning implies that the USA furthers a two-state solution. Other
chains of reasoning support this conclusion. The USA does not want to negotiate
with Hamas, since Hamas lends no support to a two-state solution, and denies the
possibility of a peace process that could bring about a two-state solution. The USA
drops the demands of the Palestinians, and welcomes Israel, which also drops the
demands of the Palestinians, since the Palestinians themselves will not lead back
to the peace process, which could bring about the two-state solution.
Nevertheless, the sample article gives an inconsistent, unbalanced view
(Heider 1946) of the US position. The unbalance hinges on two inconsistencies.
The Palestinians demand a settlement freeze that could open up the peace process,
but they will not lead the path back to negotiations, according to the author of the
example article. The USA welcomes the Palestinians, but also drop their demands.
Given these inconsistencies, one may also argue that the USA now rejects a twostate solution. For example, the USA welcomed Palestinians who will not lead
back to the peace process themselves. More importantly, they dropped Palestinian
demands although these demands for a settlement freeze could have opened the
peace process, which could have brought about a two-state solution.
13
14
In summary, a semantic network analysis aims at inferring political meanings from texts by highlighting the chains of reasoning in texts, as well as their
consistency or inconsistency. The semantic network analysis of the sample article
clearly reveals that it throws the ball into the Palestinian court by portraying the
Palestinian demands for a settlement freeze as the major obstacle for the peace
process, rather than the Israeli settlements.
Automation using semantic rules on top of an ontology, POS-tags, syntax

dependency trees and a sentiment analysis of predicates
We will now show how grammatical analysis can be used to automatically extract
part of the network as extracted by human coders. In particular, we will extract
citations (sources) and semantic subject/predicate/object triples. Grammatical
analysis yields `syntax trees, i.e., graphs containing the grammatical relations
between the words of a sentence. For this example, we used the freely available
Stanford parser to parse the sentences listed above (Klein and Manning 2003). In
other cases we have used the (also freely available) Alpino parser for Dutch with
similar techniques (Van Atteveldt et al. 2008; Van Noord 2006).
The key intuition for using syntax trees is that these trees are closer to the
(semantic) relation we wish to measure than the raw words of the sentence. As
an example, consider the sentences John hits Mary and Mary, who has been the
victim of domestic violence before, was hit by John. Both sentences express a hitting relation between John (the hitter) and Mary. However, the surface structure
is very different, with many (for this relation) irrelevant words in between John
and Mary in the second example and the reversed order of John and Mary. As will
be shown below the grammatical structure of these sentences will make it clear
that the relative clause (, who has been.) is not central to the expressed relation
and that the second sentence is in the passive voice.
We use the grammatical structure of the text by defining rules that match
specific patterns in the syntax trees. The concepts occurring in the relevant parts
of the patterns are then translated to semantic roles between these concepts. To
illustrate this, Figure 5 shows the annotated parse tree of the fifth sentence from
the example above. The words in italics are the words from the text, with the
labels on the edges indicating the grammatical relations between them. President
Abbas, for example, is the subject of stated, while the whole sub-tree under would
create is the complement of that verb.2 As can be seen from this graph, irrelevant
2. Note that noun phrase (NP) chunks were collapsed to simplify the graph. Normally, each
word would be a single node, with for example Abbas being the subject of stated and President
a modifying node under Abbas.
modifiers such as repeatedly, only, and since then are no longer in between the
predicates and their subjects, bringing the grammatical structure indeed closer to
our intended semantic structure. Moreover, grammatical relations such as subject
are related (but not identical) to the semantic relations, giving hope that moving
from grammar to semantics might be doable.
Stated (that)
preposition
since
subject
President abbas
Citation: source
modifier
repeatedly
object
then
complement clause
would create
Citation: quote
Triple: predicate
modifier
only
subject
a complete israeli
settlement freeze
Triple: subject
object
the conditions
Triple: object
preposition
for
object
a return to the
negotiating table
Figure 5. Semantic tree of sentence 5:

Since then, president Abbas stated repeatedly that only a complete Israeli settlement
freeze would create the conditions for a return to the negotiating table.
In order to move from grammar to semantics, we have defined a (relatively) small

number of rules that match patterns on the syntactic tree. Figure 6 contains a list
of the four rules that are used in the examples here. In the current example, Rule 1
matches the stated verb as it is a speech act verb and it has a subject and complement. Thus, a Citation is created with President Abbas as source and would create
(and all nodes below it) as quote. Rule 2 matches the verb would create, as it is not
a speech act and has a subject and object. Thus, a Triple is created with the settlement freeze as subject, the conditions (again including underlying nodes) as object,
and the verb could create as predicate.
If we use the same ontology to identify the concepts of interest, and if we can
detect that create is a positive relation, we can reduce this citation containing a
triple to a s: xRy relation [Palestine: SettlementFreeze + PeaceProcess].
15
16
Rule 1: Citation
Pattern:
a speech act verb (state, say, ) with a subject child node and a
complement child node
Citation(source = subject, quote = complement)
Result:
Rule 2: Active verbs

Pattern:
a non-speech act verb with a subject child and an object child
Result:
Triple(subject = subject, predicate = verb, object = object)
Rule 3: Citation from gerund
Pattern:
a speech act gerund (stating, saying, ) with a complement parent
node that has a subject child node and a complement clause child node,
Result:
Citation(source = subject of complement, quote = complement clause)
Rule 4: Action Nouns
Pattern:
an action noun (attack, policy) with a possessive child and a
preposition object grandchild
Triple(subject = possessive, predicate = noun, object = object
of preposition)
Result:
Figure 6. Selection of pattern rules for detecting semantic roles.

attacked
Triple1: Predicate
subject
Mr. Gaylard
Triple1: Subject
Citation: Source
complement
policy
Triple1: Object
Triple2: Predicate
possessive
Israels
Triple2: Subject
object
multiword
siege
saying
compl. clause
preposition
towards
object
Gaza
Triple2: Object
strengthened
Citation: Quote; Triple3: Predicate
subject
it
Triple3: Subject
auxilliary
had
object
Hamas
Triple3: Object
Figure 7. Semantic tree of Mr Gaylard attacked Israels siege policy towards Gaza,
saying it had strengthened Hamas.
Figure 7 shows a more complicated example from an actual newspaper article:

Mr Gaylard attacked Israels siege policy towards Gaza, saying it had strengthened
Hamas. This tree sets off rules: Rule 2 matches both active verbs strengthened
and attacked. Rule 3 matches the gerund saying, finding as its source the subject
of its parent node attacked and as its quote the clause below strengthened. Rule 4
matches the policy, using the prepositional object Gaza as object and the possessive Israels as subject. This yields the annotations Citation and Triple1 to Triple3
displayed in the syntax tree. In order to create a semantic network from these
annotations we would need a suitable ontology to link Mr. Gaylard to the UN
and Gaza to Palestinians. Moreover, we need sentiment analysis to determine that
siege policy and attack are negative while strengthening is positive. Finally, we need
to use anaphora resolution to determine that the it in this sentence refers back to
Israels policy (Lappin and Laess 1994; Van Atteveldt et al. 2008). This yields three
semantic roles: [UN policy], [Israel Palestinians], and [UN: policy + Hamas].
Note that this example showcases another complexity in extracting a semantic
graph from language: graphs are by definition first order, meaning that relations
cannot themselves be used as nodes in another relation. However, in natural language relations are frequently nested, as in our example of Mr Gaylard attacking
Israels policy against Gaza. To reduce this complex network to a normal graph
we need to resolve these containments using transitivity rules based on cognitive-consistencytheories discussed above. In this case, we would conclude that
MrGaylard is against Israel (since he disagrees with their policy) and in favour of
Gaza (since he disagrees with a policy detrimental to them).
These steps entail a substantive interpretation of the implications of statements and a move from the manifest to the latent content of the text. Since we can
formally describe the rules for these interpretations this is much nicer, however,
than asking human coders to draw such inferences, since it is sometimes difficult
to keep political knowledge (and bias) away from the interpretation.
Summary
Starting from old ideas about politics and political language this chapter explored
whether the occurrence of concepts, their co-occurrence, and the relationships
between them can be extracted automatically so as to infer the political meanings underlying a text. From a theoretical perspective, these three objectives correspond with the automation of first-order agenda setting, second-order agenda
setting and semantic network analysis. The latter is not only concerned with the
extraction of issue positions of actors from texts, but simultaneously with the
extraction of other political relationships, such as conflict or cooperation between
actors, success or failure of actors, consequences of issues for actors, causal relationships between issues, and so on.
By showing what information the different methods (association analysis,
manual coding, syntactic parsing) extract from a single sample text on the Middle
17
18
East conflict, the chapter illustrates how these different methods show the semantic network expressed in this text in different levels of detail. By using pattern
matching on the automatically parsed syntax trees, it showed that automation of
semantic network analysis can proceed beyond word counts and co-occurrence.
It also illustrated the complex patterns originating from single sentences and the
additional techniques required to move from extracted syntactic roles to a full
semantic network.
References
Ansolabehere, S., and S. Iyengar. 1995. Going Negative: How Attack Ads Shrink and Polarize the
Electorate. New York: Free Press.
Antoniou, G., and F. van Harmelen. 2004. A Semantic Web Primer. Cambridge: MIT Press.
Budge, I., and D.J. Farlie. 1983. Explaining and Predicting Elections: Issues Effects and Party
Strategies in Twenty-Three Democracies. London: George, Allen and Urwin.
Budge, I., H.D. Klingemann, A. Volkens,, J. Bara, and Tanenbaum, E. 2001. Mapping Policy
Preferences: Estimates for Parties, Electors, and Governments, 19451998. Oxford: Oxford
University Press.
Budge, I., D. Robertson, and D. Hearl. 1987. Ideology, Strategy and Party Change: A Spatial
Analysis of Post-war Election Programmes in 19 Democracies. Cambridge: Cambridge University Press. DOI: 10.1017/CBO9780511558771
Cappella, J.I., and K.H. Jamieson. 1997. Spiral of Cynicism: The Press and The Public Good. New
York and Oxford: Oxford University Press.
De Tocqueville, A.C.H. C. 1951 [1835]. De la Dmocratie en Amrique. Paris: Gallimard.
Deetjen, G. 1977. Industriellenprofile in Massenmedien: Ein neuer Ansatz zur Aussagenanalyse.
Hamburg: Verlag Hans-Bredow-Institut.
Dixon, R.M.W. 1992. A New Approach to English Grammar, Based on Semantic Principles.
Oxford: Clarendon.
Dixon, R.M.W. 2005. A Semantic approach to English Grammar. Oxford: Clarendon.
Elster, J. 2009. Alexis de Tocqueville: The First Social Scientist. Cambridge: Cambridge University
Press. DOI: 10.1017/CBO9780511800429
Hayes, D. 2005. Candidate qualities through a partisan lens: a theory of trait ownership. American Journal of Political Science 49(4), pp. 908923. DOI: 10.1111/j.1540-5907.2005.00163.x
Heider, F. 1946. Attitudes and cognitive organization. The Journal of Psychology, 21(1), pp. 107
112. DOI: 10.1080/00223980.1946.9917275
Hetherington, M.J. 1996. The medias role in forming voters national economic evaluations
in 1992. American Journal of Political Science 40(2), pp. 372-395. DOI: 10.2307/2111629
Jones, B., and F. Baumgartner. 2005. The Politics of Attention: How Government Prioritizes Problems. Chicago: University of Chicago Press.
Klein, D., and Ch.D. Manning. 2003.Fast exact inference with a factored model for natural
language parsing. InAdvances in Neural Information Processing Systems 15 (NIPS 2002),
Cambridge, MA: MIT Press, pp. 310.
Kleinnijenhuis, J. 2008. Reasoning in economic discourse: A network approach to the Dutch

Press. In K. Krippendorff and M.A. Bock (eds), The Content Analysis Reader, pp. 430442.
Thousand Oaks: Sage.
Kleinnijenhuis, J., and P. Pennings. 2001. Measurement of party positions on the basis of party
programmes, media coverage and voter perceptions. In M. Laver (ed.), Estimating the Policy
Positions of Political Actors. London and New York: Routledge, pp. 162182
Kleinnijenhuis, J., A.M.J. Van Hoof, D. Oegema, and J.A. De Ridder. 2007. A test of rivaling
approaches to explain news effects: News on issue positions of parties, real world developments, support and criticism, and success and failure. Journal of Communication 57(2),
pp. 366384. DOI: 10.1111/j.1460-2466.2007.00347.x
Krippendorff, K. 2004. Content Analysis. Thousand Oaks: Sage.
Lappin, S. and Laess, H.J. 1994. An algorithm for pronominal anaphora resolution. Computational Linguistics, 20(4), pp. 535561.
Lipset, S., and S. Rokkan. 1967. Cleavage Structures, Party Systems, and Voter Alignments: An
Introduction. New York: The Free Press.
McCombs, M.E. 2004. Setting the Agenda: The Mass Media and Public Opinion. Cambridge:
Polity Press.
Mena-Montes, N. 2010. Un estudio sobre la inmigracin (20002008): La construccin de la
agenda (Agenda Building) y la evolucin de los encuadres (Frame Building) en el discurso
poltico-parlamentario, meditico y sus referencias a la opinin pblica (with an extended
English summary). Madrid: Universidad Rey Juan Carlos (PhD dissertation).
Noelle-Neumann, E. 1980. Die Schweigespirale: ffentliche Meinung -- unsere soziale Haut.
Frankfurt am Main: Ullstein.
Osgood, C.E., S. Saporta, and J.C. Nunally. 1956. Evaluative assertion analysis. Litera 3,
pp.47102.
Patterson, T.E. 1993. Out of Order. New York: Knopf.
Petrocik, J.R. 1996. Issue ownership in Presidential Elections, with a 1980 case study. American
Journal of Political Science 40, pp. 825850. DOI: 10.2307/2111797
Richardson, J.E. 2004. (Mis) Representing Islam: The Racism and Rhetoric of British Broadsheet
Newspapers. Amsterdam: John Benjamins Publishing Company. DOI: 10.1075/dapsac.9
Rogers, E.M., J.W. Dearing, and D. Bregman. 1993. The anatomy of agenda-setting research.
Journal of Communication 43(2), pp. 6884. DOI: 10.1111/j.1460-2466.1993.tb01263.x
Ruigrok, N., and W. van Atteveldt. 2007. Global angling with a local angle: How U.S., British,
and Dutch newspapers frame global and local terrorist attacks. The Harvard International
Journal of Press and Politics 12, pp. 6890. DOI: 10.1177/1081180X06297436
Sanders, D., and N. Gavin. 2004. Television news, economic perceptions and political preferences in Britain, 19972001. The Journal of Politics 66(4), pp. 12451266. DOI:
10.1111/j.0022-3816.2004.00298.x
Schumpeter, J. 1950 (published first in 1943). Capitalism, Socialism and Democracy. New York:
Harper and Row.
Severin, W., and J.W. Tankard. 2005 (5th ed.). Communication Theories: Origins, Methods, Uses.
New York: Addison-Wesley / Longman.
Shah, D.V., M.D. Watts. D. Domke, and D.P. Fan. 2002. News framing and cueing of issue
regimes: Explaining clintons public approval in spite of scandal Public Opinion Quarterly
66(3), pp. 339370. DOI: 10.1086/341396
19
20 Jan Kleinnijenhuis and Wouter van Atteveldt
Soroka, S. 2006. Good news and bad news: Asymmetric responses to economic information
Journal of Politics 68(2), pp. 372385. DOI: 10.1111/j.1468-2508.2006.00413.x
Tomz, M., and R.P. van Houweling. 2008. Candidate positioning and voter choice. American
Political Science Review 102(3), pp. 303318. DOI: 10.1017/S0003055408080301
Tversky, A. 1977. Features of similarity. Psychological Review 48(4), pp. 327352. DOI: 10.1037/
0033-295X.84.4.327
Van Atteveldt, W. 2008. Semantic network analysis. Techniques for extracting, representing and
querying media content. PhD Thesis. Amsterdam: Vrije Universiteit Amsterdam.
van Atteveldt, W. 2008. Semantic Network Analysis: Techniques for Extracting, Representing and
Querying Media Content. Charleston, SC: BookSurge Publishers.
Van Atteveldt, W., J. Kleinnijenhuis, and N. Ruigrok. 2008. Parsing, semantic networks, and
political authority: Using syntactic analysis to extract semantic relations from Dutch newspaper articles. Political Analysis, 16(4), pp. 428446.
van Cuilenburg, J.J., J. Kleinnijenhuis, and J.A. de Ridder. 1986. Towards a graph theory of
journalistic texts. European Journal of Communication 1, pp. 6596. DOI: 10.1177/
0267323186001001005
Van Noord, G.-J. 2006.Atlastparsingisnowoperational. In Mertens, P., C. Fairon, A. Dister,
and P. Watrin (eds). TALN06. Verbum Ex Machina. Actes de la 13e conference sur le traitement automatique des langues naturelles, pp. 2042.
Westholm, A. 1997. Distance versus direction: the illusory defeat of the proximity theory of
electoral choice. The American Political Science Review 91(4), pp. 865883. DOI: 10.2307/
2952170
Wittgenstein, L. 1922. Tractatus Logico-Philosophicus: Logisch-philosophische Abhandlung. London:
Kegan Paul.
Wittgenstein, L. 1953. Philosophical Investigations. Oxford: Blackwell.
part i
Computational methods
forpoliticaltextanalysis
Introduction
Piek Vossen
Political language is one of the most challenging text types to analyze. In debates,
politicians use language with great rhetorical skill. For computational analysis,
political language is a genuine challenge. Natural language processing (NLP) by
computers can be used to automatically analyze text and to derive implications
from it, but all technology is limited to information that is structurally and statistically observable. There are implications that computers can draw from analyzing
vast amounts of text, which people cannot do simply because they cannot store
exact statistical data and cannot read so fast. Computers can, for example, detect
trends in word usage across time and across different groups of (political) speakers
that signify changes in political discourse. However, we also know that there are
vast amounts of implications that humans can draw even from the smallest piece of
text, but computers cannot do this. People can because they have an understanding
of the complex social relations between the participants in the debate, have rich
and complex background knowledge about the world we live in and are extremely
experienced and sensitive to the use of language within such contexts. It is one of
the exciting aspects of the field of text mining to see how far we can get in drawing implications from (political) text. Part I of this volume contains 6 articles that
illustrate the possibilities and limitations of applying computational techniques to
the analysis of political text. It is not a comprehensive overview, but the following
chapters show some of the main issues that are currently under discussion.
In principle, the units and aspects of language can vary from individual words,
to full text and collections of text, and from plain statistics to party issue positions
and rhetoric. Table 1 gives an overview of the possible units of analysis (cf. first
column) and their types of analysis (cf. first row).
Research and development is done on all these aspects but progress and the
complexity of such efforts vary a lot. Statistics can be derived easily for any structure,
but automatic analysis is more complex as we move on the scale from statistics to
world knowledge, and fewer systems are available that can do the job. In this book
we see examples of NLP techniques centered to the left side of the table, whereas
deeper qualitative approaches at the right side are necessarily restricted to a single
or a small set of documents to be analyzed by humans (see Wesley, this volume).
24
Piek Vossen
Table 1. Units of text and types of text analysis in the field.

Statistics Structure Meaning Polarity Position Rhetoric World
knowledge
words
phrases
sentences
+/
paragraphs
complete discourse
or document level
+/
text collections
+/
In terms of methods, NLP has seen an impressive development over the past
decades from rule-based and knowledge-rich systems, to machine-learning
approaches, and most recently, to hybrid solutions. For various reasons, statistical
and machine-learning methods are more accessible and widely used. One reason
is their success over the traditional rule-based systems. Another reason is the lightweight and shallow processing of the text, which can be applied to large volumes.
It has been shown that statistical and machine-learning NLP can, to a reasonable
extent, predict party positions along political dimensions on the basis of language
used by politicians. This book includes several examples of how this can be done.
In Chapters 2 (Collette and Ptry) and 5 (Hirst, Riabinin, Graham, BoizotRoche and Morris), we see how the words of texts can be used to find party positions. In these approaches, the assumption is made that similar texts tend to use
similar words and that there is no need to preserve the structure and complex
composition of texts. The words that make up the text are the features that are
machine learned to make predictions. Collette and Ptry compare simple wordfrequency methods of the widely-used programs Wordfish and Wordscores to
English and French versions of Canadian party manifestos and evaluate them
against expert surveys on party positions. They try to measure degrees of influence of different languages on party positioning and also consider the effects of
word stemming (reducing the feature-space). The programs associate words and
their frequencies with the party manifestos of one election and compare these with
the frequencies in the manifestos of another election. Collette and Ptry show that
this works equally well for English and French parliamentary debates in Canada,
despite the different morphological properties of these languages. They also note
that Wordscores outperforms Wordfish and that stemming words does not lead
to significant effects.
Chapter 5 represents an interesting contrast with this paper. Hirst et al. also
try to discover party positions in Canadian politics (liberal versus conservative)
but use a Support-Vector Machine (SVM) as a model and apply their analysis to
Part I. Introduction
the English and French debates rather than to party manifestos. Hirst et al. come
to the remarkable conclusion that their SVM classifier does not learn the language
of the political position but merely the language of defense and attack. In their
corpus, the liberals and conservatives swap places from opposition to government
and vice versa. A classifier trained with the opposition language identifies the
opposition regardless of the partys political status and the other way around for
the language of the governing party. The results hold for both English and French.
This chapter demonstrates that texts consist of many layers of information and it
is dangerous to associate bags-of-words with any type of labels in classifiers, since
we do not know what the classifier actually learns. In other words, we do not know
which words belong to which layer of information. This is a genuine risk of any
machine-learning approach, which can only be dealt with by rigorous testing and
evaluation on many different data sets.
Another pair of chapters tries to extract positive/negative attitude or sentiment from heterogeneous types of text. In Chapter 3, Gryc and Moilanen compare different types of linguistic attitude indicators and social network data to
derive sentiments centered around Barack Obama during the 2008 US elections.
They apply their methods to 700 blog posts that are classified by crowd sourcing.
Different feature sets are derived from the blog post (social network features, sentiment words, bags-of-words) and different classifiers are trained. Results are moderate, with the bags-of-words approach (using a large feature set) working best but
social network analysis appearing to contribute additional evidence. Combining
different classifiers through voting gives the best results, which is a well-known
phenomenon in machine learning. Chapter 6 is closely related to this work: here,
Grijzenhout, Jijkoun and Marx compare various techniques to determine subjectivity and polarity of paragraphs in Dutch parliamentary debates. They split the
problem into determining: (1) the subjectivity of a paragraph and (2) the polarity
of the subjectivity found. They compare different types of mathematical models
for learning classifiers from training data and contrast the results with algorithms
based on subjectivity lexicons. Both chapters show that state-of-the-art approaches
to sentiment analysis (both machine-learning and lexicon-based) give reasonable results for political topics in various types of text, such as blogs and debates.
Nevertheless, sentiment analysis for negative or positive attitude is rather onedimensional compared to a complete analysis of the meaning and implication of
political text.
Chapter 4 can be seen as an attempt to use similar techniques to perform a
more complex task, namely to model the usage of the concept outsiders in the
Swedish political debate. Dahlberg and Sahlgren use Random Indexing to measure concept-shifts. The basic idea is that statistics on the surrounding words tell
you something about the meaning of a target word or concept. Whereas in the
25
26 Piek Vossen
previous techniques presented in this volume, sets of words are used to model
larger text fragments associated with positions or sentiments, in this research the
surrounding words (the word space) characterize the word outsider itself. In a
first step, the language surrounding the concept outsider is learned from official
documents of different parties. Next, the language used in a large collection of
blogs is analyzed and compared with the outsider language of the parties. In this
study, the analysis is complicated by lack of diachronic data. However, the outsider
language of the parties tends to be similar and provides some evidence that the
Conservative Moderate Party may have introduced connected concepts such as
unemployment to the word space.
The chapters on computational methods show a strong tendency for statistical or quantitative approaches rather than for deeper qualitative approaches.
In the light of Chapter 1, by Kleinnijenhuis and van Atteveldt, this kind of text
analysis is relatively shallow. Even so, the conclusions and results of these shallowquantitative techniques are still not without controversy. Results are moderate
and probably not stable across different types of texts. Furthermore, Hirst et al.
clearly show that we do not know what is learned since text, and definitely political
text, comprises many different layers of information that may have been mixed
up by the statistical analysis. However, it remains to be seen if deeper text analysis and more comprehensive approaches, as described by Kleinnijenhuis and van
Atteveldt, can do a better job. For instance, the Net Method also leaves out many
details and works by virtue of large volumes of news texts to wash out statistical
value from noisy data. In order to become aware of the limitations and to evaluate
the adequacy of computerized methods one should carefully consider and account
for what is ignored.
To build a bridge between the (dis-)advantages of automated text analysis
and methods for qualitative, interpretive analysis, Part I ends with a discussion
of the fundamental differences between qualitative and quantitative methods for
analyzing political texts that are the subject of Parts II and III (Wesley, this volume). Even though quantitative methods are often computerized and therefore
assumed to be more objective, Wesley argues for qualitative approaches in which
the interpretation of the researcher plays a role even when it leads to a bias in the
interpretation of results. He claims that interpretive-subjective methods can result
in productive insights as long as they are applied rigorously, following specified
guidelines and choices are properly documented. Although not following the CDA
paradigm, Wesleys chapter indicates a need for text analysis beyond quantitative
premises. His approach builds a bridge to the chapters on qualitative discourseanalytic methods presented in Part II.
chapter 2
Comparing the position of Canadian

politicalparties using French and English
manifestos as textual data
Universit Laval, Department of Political Science
Recently, computer-assisted, quantitative methods have been developed to

position political parties. These word-based textual analysis techniques rely
exclusively on the relative frequency of words. As such they do not necessitate the knowledge of any particular language to extract policy positions from
texts. However, different languages have different word distributions and other
syntactic idiosyncrasies. These differences might provoke word-based textual
analysis techniques to extract noticeably different positions from parallel texts
that are similar in every aspect except language. How crippling is this potential
disadvantage when comparing political texts written in different languages? It
is this chapters objective to determine the effect of language on the two word
frequency methods Wordscores and Wordfish by comparing the policy positions
of Canadian parties as extracted from their English and French party manifestos.
Word-based parallel content analysis

Over the past thirty years, the methods employed by researchers to locate political parties have evolved from hand-coded methods, such as the well-known
Comparative Manifesto Project (CMP) (Budge et al. 2001; Klingemann et al.
2007), to expert surveys (Castles and Mair 1984; Huber and Inglehart 1995; Laver
and Hunt 1992) to dictionary methods (Kleinnijenhuis and Pennings 1999; Laver
and Garry 2000; Ray 2001). New computer-assisted, quantitative methods for
extracting political party positions on the left-right axis or other policy dimensions from political texts have been a useful addition to the researchers toolbox.
They rely on objective textual data, they can be used to a nearly unlimited flow of
data, and they make it possible to isolate policy preferences from behaviour (see
Benoit and Laver 2007b; Laver and Garry 2000; Marks et al. 2007). One advantage
28
of word-based textual analysis techniques is that, since they rely exclusively on the
relative frequency of words, they do not necessitate the knowledge of any particular language to extract policy positions from texts. However, different languages
have different word distributions and other syntactic differences. These differences
might provoke word-based textual analysis techniques to extract noticeably different positions from parallel texts that are similar in every aspect except language.
Word-based techniques, such as Wordscores (Laver et al. 2003) and
Wordfish (Slapin and Proksch 2008), analyse the distribution of words in political
texts to extract policy positions from them. We call these techniques word-based
because the analytical units are the words in a text, not paragraphs, sentences,
locutions or topics. This particularity has two main advantages. By chopping texts
into words, word-based techniques gain the advantage of analytical simplicity,
because words can be automatically identified and treated without human intervention.1 Second, since words are treated like quantitative data, the knowledge of
a language is no longer necessary to extract and then compare policy positions
from texts written in different languages.
The disadvantage of word-based techniques is that they do not take into
account the meaning or the grammatical structure of sentences and words that
make them up. Focusing exclusively on the relative frequency of mention of words
can lead to linguistic nonsense when the logic is pushed to its limits. As an extreme
illustration, it is possible to extract a policy position from a random bunch of
words that no human reader could make sense of, or from a freely reorganized
text in, say, alphabetical order. It is impossible to measure the positive or negative
direction of a policy preference in a text (Monroe et al. 2008). For example, the
meaning of the sentence We will raise taxes is the exact opposite of We will not
raise taxes. But if we cut these sentences into separate words we (2 times) will
(2) not (1) raise (2), and taxes (2), the difference between the two is not, a
relatively meaningless word which is likely to be overlooked. The difference in the
meanings of the two sentences will be blurred as a result.
How crippling is this potential disadvantage when comparing political texts
written in different languages? It is this chapters objective to determine the effect
of language by comparing the policy positions of parties in recent Canadian elections extracted from English and French party manifestos. We do this by checking whether two different methods for automated text analysis, Wordscores and
Wordfish, extract the same policy positions on the left-right axis from parallel
1. In practice things are more complicated and experience showed us that, for example,
hyphenated locutions can be sometimes treated as a single word or as separated words
depending on the method used to produce a frequency matrix.
Chapter 2. Comparing the position of Canadian political parties
texts. Parallel texts are original documents written in different languages, not
translations. They can be used to benchmark automatic translation quality (see
Jian-Yun et al. 1999 for applications of parallel texts).
Do Wordscores and Wordfish extract the same policy positions on the leftright axis from parallel documents? In theory, there is no reason to believe
that they would not. Parallel documents are rigorously the same. Their format
is identical, they include the same topics and a bilingual reader will consider
those documents to be almost exactly the same. Studies analyzing parallel textual data are interesting because they give an opportunity to test the validity and
the reliability of word-based textual analysis methods in a more rigorous way
than repeated studies focusing on a single language and/or a single party system. Wordscores has been tested with languages other than English, such as
Dutch (Klemmensen et al. 2007), German (Hug and Schulz 2007; Magin et al.
2009; Bruninger and Debus 2008), and French (Laver et al. 2006). But we could
find only one study comparing Wordscores results using parallel texts as input.
Debus (2009:5354) uses Wordscores to compare Flemish and French coalition
agreements in Belgium and finds no significant difference between them for economic policy positions. In their analysis of European Parliament speeches Slapin
and Proksch (2008) compare Wordfish results for speeches in English, French
and German. They find remarkable similarities between languages (English and
French especially):
The comparison of the results across languages suggests that the position estimation technique is in fact highly robust to the choice of language (the correlation coefficient is 0.86 or higher). The highest correlation is between positions
estimated from the English and French translations. These two languages are so
similar to each other with regard to the information contained in words that they
produce virtually identical position estimates. (Proksch and Slapin 2009:13)
It should be noted that due to the large number of official languages spoken in
the European Union, speeches in the European Parliament are delivered in one
language and then translated. Slapin and Proksch relied on translations instead of
original texts. It is unclear whether the relatively high level of similarity between
texts in different languages was achieved because, or in spite, of the fact that they
were translations. One way to clarify this is to extract party positions from parallel
documents which original versions are written in more than one language.
Some specific features of languages can have a significant impact on the distribution of words in a text. French and English differ on many levels: syntax, grammar, and style (see Lederer 1994; Vinay and Darbelnet 2003 [1977]). An important
syntactic difference between French and English is the use of articles. In French,
they are more frequent than in English, because they are quasi-mandatory before
29
30
nouns. We can expect to find some common articles (de, du, la, les, etc.) to have a
very high frequency, higher than the equivalent in English.
Looking at grammar, we find several significant differences. In French, grammatical gender results in a duplicate set of adjective, pronoun and article forms
one for masculine and one for feminine nouns. French texts tend to have relatively
higher differentiations of those words when compared to English and that means
expressing the same concepts with more words that are relatively less frequent
than in English. While in English nouns may be singular or plural, in French
adjectives can also be singular or plural, in addition of being masculine or feminine. So it is possible to have four word forms to express the same concept instead
of one in English. As a simple illustration, take the adjective pretty in English.
In French it can be declined in masculine singular form (beau), in feminine singular (belle), in masculine plural form (beaux), and feminine plural (belles)
depending on the object. Inevitably the English word pretty will be more frequent in a text than any one of the four French equivalent adjectives.
Another important difference between French and English is the verb tense
system. In English, there are two forms in the present tense, one for the third person singular (he/she/it) and one for the rest, while most of the time in past tense
you have only one form. The future, conditional and subjective tenses all have only
one form, while French generally has five forms, one for each person plural (we/
you/they) and two for the singular persons (either I/you or I/he-she-it), depending
on the verb family. Moreover those forms are applied differently to future, conditional, subjunctive and past tenses. Therefore, the same verbs tend to be declined
in more different words in French, modifying again the word distribution.
One consequence of these differences will appear in the number of words in
a text. French texts have more words than English texts. Unsurprisingly, between
2000 and 2008 manifestos from major Canadian parties contain 11,717 different
words in French, and 8,966 in English. The average frequency of words is higher in
English than in French. On the surface, the average frequency of words is higher
in French (19.1) than in English (18.7), but the standard deviation is much larger
in French (218) than in English (176). Thirteen French words (du pour, un, en,
d, l, le, a, des, la, et, les, de), many of them articles, appear more than 2,000 times,
while in English only eight words appear more than 2,000 times (for, will, in, a,
of, to, and, the). When we exclude those words the mean frequency becomes 12.8
in French and 14.3 in English. We also find more unique words (words used only
once) in French. There are 4,625 unique words in French and 3,361 in English.
Methodology
In our study, we test Wordscores and Wordfish on Canadian parallel manifestos
and compare the results on the left-right dimension with expert surveys to estimate the cross-language reliability of these two techniques. Canada, as an officially
bilingual country, is a pertinent case for this test. At the federal level, manifestos
from major parties are bilingual and both versions are considered official and
public. With the exception of the Bloc Qubcois, Canadian party manifestos can
qualify as perfect parallel texts.2 In addition, Canada has clearly identified polarized parties the New Democratic Party (NDP) and the Conservative Party on
the left-right dimension (Irvine 1987; Johnston 2008). It is necessary to have a
right and a left end point in the Canadian federal partisan spectrum in order to
use Wordscores and Wordfish because both these techniques need to set values
on reference texts.3
The analyses of both Wordscores and Wordfish are based on a frequency
matrix. The frequency matrix lists the words used in all the documents and how
frequent they occur in each document.4 There is a debate over the merits of word
stemming (reducing words to their root form) and removing stopwords (the, who,
that, le, qui, quoi, etc.) with computer-assisted content analysis methods (see Lowe
2008). Stemming words and removing stopwords tends to reduce the size of texts
and word diversity. That is supposed to lead to better results as meaningless words
are removed and family of words reduced to stems. On the other hand, these operations artificially skew the word distribution in a text that could distort results.
Although word stemming is a common practice with Wordfish, we will compare
results with non-stemmed and stemmed texts. This comparison will be a p
ertinent
2. Canadian parties publish their platforms more or less simultaneously during election campaigns. One exception was the Liberals Green Shift manifesto which was published well in
advance of the 2008 election. The NDP, the Liberals and the Conservatives claim that the English
and the French versions of their manifestos are original documents, not translations. This claim
makes political sense, although it is impossible to verify. In any case, political parties know
that any discrepancy between the English and the French versions of their manifesto would be
advertized by the media and that this would undermine their credibility. The Bloc qubcois
does not produce sufficiently reliable English versions of its manifestos to qualify as parallel
texts. The Bloc has been excluded from this analysis.
3. Contrary to what Slapin and Proksch (2008:708) state, we need reference texts with Wordfish.
Among the parameters, the user has to set omega scores for references texts representing both
extremes of the dimension on which political texts will be located.
4. The frequency matrix from which data are taken has been produced with Will Lowes jfreq
program, available at http://www.williamlowe.net/software/ [Accessed November 3, 2010].
31
32
addition in the discussion. Jfreq has built-in stemming and stopword removal
options that have been used to create the different matrixes needed.5
The corpus is composed of Canadian manifestos from the 2000, 2004, 2006,
and 2008 federal elections. The relatively short period of time will minimize
the problem of political vocabulary change, a concern for both Wordscores and
Wordfish (Budge and Pennings 2007; Slapin and Proksch 2008). Three major parties are included in the analysis: the New Democratic Party (NDP), the Liberal
Party and the Conservative Party.6 Original manifestos have been tailored and
standardized to facilitate content analysis. Messages from the leaders, financial
annexes, tables, and quotations from other works (newspaper clips or OECD
Reports for example) have been excluded from the original texts. For Wordfish,
we followed Proksch and Slapins (2008) recommendation and eliminated unique
party words (words that are used by only one party) from the texts analyzed.
Canadian expert surveys

Expert surveys will serve as a benchmark to compare the face validity of the
Wordscores and Wordfish estimates. Expert surveys are widely used to locate
political parties in a policy space and often serve as a benchmark for textual analysis (Benoit and Laver 2007b; Keman 2007; Laver and Garry 2000; Marks et al.
2007; Ray 2007; Volkens 2007; Whitefield et al. 2007).7 This approach was made
popular thanks to a well publicized study by Castles and Mair (1984) who asked
political scientists in 17 countries to locate political parties in their own country
on an 11-point left-right scale. This was followed by a more ambitious study of
left-right party position in 42 countries (Huber and Inglehart 1995). These two
studies have inspired a large body of country-specific studies using expert surveys
to locate political parties on a range of issues (for a sample see Laver 1998; Laver
and Hunt 1992).
5. To remove French stopwords, we used the DotSEO list available at http://www.dot-seo.com/

french-stop-words/ [Accessed November 3, 2010]. We had to tailor the list to fit this research.
The modified list can be provided upon request by the authors.
6. The Reform Party/Canadian Alliance is not included because it no longer exists. It merged
with the Progressive-Conservative Party in 2004 to make the Conservative Party. The Bloc
Qubcois is excluded because it does not publish English manifestos.
7. For reviews of the data on party placement and on self placement on policy spaces using
the mass or expert survey method see Knutsen (1998); Budge (2000); Laver and Garry (2000);
Mair (2001).
In constructing our own expert survey, we replicate the well-tested methodology already used by Laver and Hunt (1992), Laver (1998), Laver and Benoit (2005),
Benoit and Laver (2007b). Our online survey questionnaire included 18 questions on a 010 scale. Respondents were asked to position the Bloc, Conservatives,
Liberals, NDP, and Greens using the 11-point scale on the left-right axis and other
policy dimensions. For all dimensions 0 represents the left position and 10, the
right position. The expert survey was conducted immediately after the Canadian
general election, held on October 14, 2008. The electronic questionnaire was sent
to every political science professor in Canadian universities. 163 experts sent back
the questionnaire out of 1,076 (15%).8 Of the total, 127 questionnaires (78%) were
filled in English and 27 (22%) in French.
Table 1. Canadian expert surveys in 2003 and 2008, 010 left-right scale.
Party
Mean
both
Conf.
interval
Mean
English
Conf.
interval
Mean
French
Conf.
interval
Laver/Benoit
2003 (s.e.)
Conservative
Liberal
NDP
8.0
5.3
2.9
7.98.2
5.05.6
2.73.1
7.9
5.2
2.9
7.78.1
5.05.4
2.73.1
8.4
5.5
3.0
8.08.8
5.25.8
2.53.6
7.48 (0.11)
6.1 (0.09)
2.7 (0.12)
Table 1 indicates how the experts who responded to our survey located the parties on the left-right policy dimension. Results from the Benoit and Laver survey
(2006:205), conducted in 2003 and included in the table, are quite similar.9 The
numbers in the first column are the mean scores for the entire sample of experts.
But what is of direct interest here is the mean score for each subsample. The mean
score for the position of the Conservative Party among English speaking respondents is 7.94 with a 95% confidence interval ranging from 7.7 to 8.1. If our sample
of experts were a random sample (which it is not) we would say that 19 times out
of 20 English speaking experts in the population located the Conservative Party
between 7.7 and 8.1 points on the left-right axis in 2008. The mean score for the
position of the Conservative Party among French speaking respondents is 8.39
with a 95% confidence interval ranging from 8.0 to 8.8. Note that the confidence
interval for French speaking experts is much larger than for English speaking
experts (this is of course due to the much smaller size of the French sample) and
that the two intervals barely overlap each other. In other words, it is likely that
French experts located the Conservative Party significantly more to the right than
English experts in 2008. The mean scores for the position of the Liberal Party and
8. The data from the 2008 expert survey can be obtained upon request from the authors.
9. They report 104 respondents, for a rate of 17% (Benoit and Laver 2006:196).
33
34
the NDP are 5.2 for English experts and 5.5 for French experts, respectively. The
small overlap between the two confidence intervals makes it once again likely that
French speaking experts in the population locate the Liberals significantly more to
the right than English speaking experts. The mean scores for the NDP differ very
little in the sample and the large overlap of the confidence intervals indicate that
the position of the NDP on a left-right axis by French speaking experts is indistinguishable from the position by English speaking experts. It should be noted
that the scores for the Conservatives, the Liberals and the NDP are all statistically
different from each other among both English and French speaking experts.
To summarize, both French and English speaking experts distinctly locate
the Conservatives, the Liberals and the NDP. French speaking experts locate the
Conservatives and the Liberals more to the right than English speaking experts,
although the difference is not large for the Liberals. There is no difference in the
way English and French experts locate the NDP.
Wordscores
The first computer-assisted method selected to locate the positions of political
parties is the Wordscores computer program developed by Michael Laver and
his co-researchers (Laver, Benoit and Garry 2003).10 Unlike the CMP and dictionary methods that treat texts as discourse to be understood and interpreted
for meaning either by a human coder or by a computer, Wordscores treats texts
(more precisely the words contained in those texts) as data containing information
about the position of the texts authors on predefined policy dimensions. Starting
from a set of reference texts whose policy positions are determined a priori, the
technique extracts data from these reference texts in the form of word frequencies
and uses this information to estimate the policy positions of virgin texts about
which nothing is known.
Wordscores has the advantage of producing a distribution of scores around
an estimated mean score. This makes it possible to calculate a standard error
and hence to establish a confidence interval around the estimated mean score.
Wordscores provides a statistical measure of how different two virgin texts are
from one another in their vocabulary. Two texts are statistically different if their
confidence intervals do not overlap. Of course, the scores are all the more valid
if one has confidence in the choice of references texts and in the measure used to
decide what their positions are on a given scale or cleavage. Laver and collaborators
10. For review of Wordscores, see Benoit and Laver (2007a, 2008); Klemmensen et al. (2007);
Lowe (2008); Martin and Vanberg (2008); Slapin and Proksch (2008); and Volkens (2007).
make several important recommendations concerning the selection and scoring

of references texts, one of them being that the virgin texts and the reference texts
must share a similar frame of reference (Laver et al. 2003:313315). In our case,
all documents are party manifestos that share the same structure. The recommendation is therefore fulfilled.
Here is a summary of how we used Wordscores to measure the positions of
the Canadian parties on policy scales.11 We first select the reference texts which
are used to represent the extreme positions on the a priori defined policy scale. It
is important that the reference texts are directly relevant to the virgin texts under
analysis. We follow the common practice of using manifestos from the previous
election as reference texts. The NDP manifestos are coded 0 (left), and the Con
servative manifestos 10 (right) for the left-right dimension.
Each virgin text (that is each party platform at each election) is then coded by
Wordscores which gives to each word in each virgin text a score between 0 and
10 according to the relative frequency of its appearance in the reference texts. For
example, if the word healthcare appears one percent of the time in the NDP
reference text, and 0.9 percent of the time in the Conservative reference text,
healthcare obtains a score equal to (0.01*0) + (0.009*10) = 0.09. By dividing the
sum of the scores associated with each word by the total number of words in a
text, we obtain an average which corresponds to the total score of the text. From
the wordscores in each reference text, we compute the textscores in each virgin
text, and then transform the virgin textscores to their original metric to be able
to locate the positions of each platform at each election in our pre-defined space.
Figure 1 and 2 show confidence intervals for English and French position estimates on the left-right dimension. The left-right ordering of the parties is consistent with the results of the expert survey. Note that the left-right ordering remains
unchanged over time and that the shifts from one election to the other have the
same direction in both the English and the French manifestos. As expected, the
NDP is positioned on the left and the Conservative party on the right with the
Liberal party somewhere between them. In Figure 1, it is impossible to statistically
distinguish Liberal and Conservative parties in 2000 and 2004, as their confidence
intervals overlap in these two elections. After 2004, the Conservative party moves
to the right while the Liberal party moves to the left and gets closer to the NDP.
The French Liberal platform which started in 2000 at the same point as the English
Liberal platform does not move as far to the left in 2006 and in 2008. With stemmed
11. There is a debate over the proper score transformation method that should be used focusing
mainly on comparability and scalability of transformed vs. raw scores (see Benoit and Laver
2008; Martin and Vanberg 2008). In this paper, we will keep the original method. This will make
comparison easier, as most researchers who use Wordscores stick with the original method.
35
5
LeftRight scale
10
15
Elections
2004 2006
b.
NDP
Conservative
Liberal
2008
2000
Elections
2004 2006
a.
2008
2000
36
NDP
Conservative
Liberal
5
LeftRight scale
10
15
Figure 1. English (a) and French (b) leftright confidence intervals of Canadian
manifestos with Wordscores.
documents, the leftward move of the Liberal party is so pronounced that the difference between the Liberals and the NDP tends to be blurred in 2006 and in 2008. The
rapprochement between Liberal and NDP manifestos after 2004 in Figures 1 and 2 is
consistent with the Canadian expert survey data of Table 1. According to Canadian
experts, the average difference between the Liberals and the NDP decreased from 3.4
points in 2003 to 2.4 points in 2008 (and the average difference between the Liberals
and the Conservatives increased from 1.4 points to 2.7 points).
English and French Wordscores estimates correlate at 97.1% with original
documents (see the Appendix for details).12 This figure goes down to 94.1% when
stemmed documents are used. Overall, English documents, original or stemmed,
have a higher correlation rate than French documents.
12. Debus (2009) does not indicate correlation levels for French and Flemish coalition agreements. Therefore we cannot establish a comparison.
5
LeftRight scale
10
15
2000
Elections
2004 2006
b.
NDP
Conservative
Liberal
2008
2000
Elections
2004 2006
a.

2008
NDP
Conservative
Liberal
5
LeftRight scale
10
15
Figure 2. English (a) and French (b) leftright confidence intervals of stemmed
Canadian manifestos with Wordscores.
Confidence intervals tend to be larger in French than in English: 3.8 against

2.5 with original documents and 3.3 against 3.0 with stemmed documents. We
would expect longer documents to be more precise, as they contain more information. French manifestos are longer in general than English manifestos and they
have more unique words and more total scored words. That means that although
French manifestos contain more words, a significant number of them have median
scores that tend to neutralize each other. For example, words like the or and are
found multiple times in all texts. They will be given a median wordscore because of
their ubiquity. Stemming has a positive impact in French by significantly reducing
the number of ubiquitous and meaningless words and thus reducing the size of
confidence intervals. The impact of stemming is more limited in English, as shown
by the confidence intervals that are larger.
37
38
Wordfish
The Wordfish method takes a different approach. Borrowing from linguistics, it
uses a nave Bayes assumption to infer the process by which words are processed
in a text.13 A text is represented as a vector of word counts (occurrences) and
individual words are assumed to be distributed at random. The probability that
each word occurs in a text is independent of the position of other words in the
text. While empirically false, nave Bayes often performs well for classification
(McCallum and Nigam 1998). Wordfish assumes the word frequencies are generated by a Poisson process. Slapin and Proksch (2008) chose this particular distribution because of its estimation simplicity. The single paramater, , is both the
mean and the variance. The functional form of the model is as follows:
ijt = exp (it + j + j it)
Where ijt is the count of word j in party is manifesto at time t, is a set of partyelection fixed effects, is a set of word fixed effects, is an estimate of a word specific weight capturing the importance of word j in discriminating between party
position, and is the estimate of party is position in election year t. Each platform
is treated as a separate party position and all positions are estimated simultaneously. That means there is no temporal constraint on the position of party is
manifesto in election t. So if a party uses words in similar relative frequencies over
time, its position will remain the same. Party movement is due to changes in word
frequencies, not to change in word signification.
Wordfish gives two sets of results. The first is an estimation of the position of
political parties on an axis that corresponds to the selected axis, with a confidence
interval for every manifesto. The second is an estimate of the position of each
word found in the selected texts. The document scores make it possible to position parties on policy dimensions and compare these positions with estimates of
other methods. For reasons of comparability, we set the same values as Wordscores
for reference texts in each dimension (0 for NDP 2004 and 10 for Conservative
2004). To calculate a 95% confidence interval, a parametric bootstrap, which is a
re-sampling method that creates new datasets, is required.
Figure 3 compares the positions of English and French manifestos with nonstemmed documents. There is a similar pattern in both languages. It is statistically
impossible to distinguish the positions of the parties in 2000, but at subsequent
elections, they move apart from each other, with the Conservatives steering to
the right and the Liberals to the left, with the NDP in between. This inversion of
13. See Lowe (2008) and Monroe et al. (2008) for a discussion over potential problems affecting
Wordfish.
5
LeftRight scale
10
15
2000
Elections
2004 2006
b.
NDP
Conservative
Liberal
2008
2000
Elections
2004 2006
a.
2008
Liberal party and NDP also present with stemmed documents in Figure 4 is
inconsistent with expert surveys and Wordscores results (both position the NDP
on the left of the Liberals). Another striking feature of Figure 3 is the difference
in the size of confidence intervals between English and French manifestos. Since
French documents are longer, we would expect them to contain more information
that logically lead to smaller confidence intervals, but results show the opposite
(7.0 against 3.6 in average). The difference is especially significant for the Liberal
party (0.46 in English against 12.6 in French), which is even more surprising,
considering the fact that Liberal manifestos are longer than other parties in both
languages. The last significant difference lies in the relative position of parties
along the scale. English manifestos tend to score between 0 and 10, close to the
original scale. French manifestos score over a range that reaches much further to
the left. In left-right terms, French manifestos are farther apart from each other.
NDP
Conservative
Liberal
20
15
10
5
0
5
LeftRight scale
10
15
20
Figure 3. English (a) and French (b) leftright scores of Canadian manifestos with
Wordfish.
39
40 Benot Collette and Franois Ptry
20
15
10
5
0
5
LeftRight scale
10
15
20
2000
Elections
2004 2006
b.
NDP
Conservative
Liberal
2008
2000
Elections
2004 2006
a.
2008
Figure 4 compares the positions of English and French manifestos with

stemmed words. Here the positioning in English and French is more similar. The
correlation between languages goes from 70.2 with original words to 98.4 with
stemmed words (see Appendix). This is comparable to results by Proksch and
Slapin (2009:19), who reported a 86% correlation between English and French
stemmed speeches. Stemming also has a positive impact on confidence interval
size (3.6 in French and 1.5 in English) with the gap diminishing between the two
languages, from 3.4 to 2.1. While scales would vary considerably in terms of size
between French and English with original documents, they have now similar sizes
with stemmed documents, ranging more or less from 10 to 10 in English from
15 to 10 in French.
NDP
Conservative
Liberal
20
15
10
5
0
5
LeftRight scale
10
15
20
Figure 4. English (a) and French (b) leftright scores of stemmed Canadian manifestos
with Wordfish.
Overall, stemming seems to have a positive impact, as confidence intervals

shrink and correlation is higher between languages. The impact is more significant
in French as we have seen. For this kind of comparison between languages, we
would follow the recommendation to proceed with stemmed documents.
Conclusion
Do grammatical differences between languages threaten the validity and the reliability of left-right party position results generated by word-based textual analysis
techniques such as Wordscores and Wordfish? Here is a summary of what we
found. Wordfish incorrectly positions the Canadian parties. The positioning of
the Liberal party on the far left of the political spectrum and the NDP close to the
center in both languages is in contradiction with expert surveys and Wordscores
estimates. Furthermore, it is also in contradiction with the perception of the
Canadian electorate and CMP results (Budge et al. 2001; Klingemannn et al. 2007).
Stemming the documents improves Wordfish results somewhat. The estimates of
the English and French stemmed documents are more highly correlated and have
smaller confidence intervals than the non-stemmed documents. However stemming and stopword removal do not solve the incorrect positioning of the NDP
and the Liberal party in either language. In this case, we cannot judge whether
grammatical differences threaten the validity and reliability of results with nonstemmed documents. In both languages they could not be considered as valid,
because the absolute position of parties is significantly different. With stemmed
documents, scales are more comparable, but estimates are still not valid. Of course,
increasing the number of cases and looking at more specific policy dimensions
could lead to different results. Maybe we are not positioning the parties on the leftright dimension after all. That is the problem: we are not sure on what dimension
we are positioning parties when using Wordfish on whole manifestos.
With Wordscores, results are more consistent with expert surveys estimates
than with Wordfish. Interestingly we see the Liberals steering to the left over time,
especially with stemmed documents. Stemming does not greatly alter Wordscores
results. Apparently Wordscores can deal easily with meaningless words, as their
ubiquity tends to give them scores close to the median that do not have a significant effect on text score. Therefore, stemming does not influence the results
significantly. The grammatical differences of the two languages do not seem to
threaten the validity and reliability of left-right party position estimates. The correlation between the estimates of the English and French manifestos is an impressive 0.97 for non-stemmed documents and 0.94 for stemmed documents. In both
41
42
English and French the positions of the manifestos shift in the same direction
from one election to the other. Compared to Wordfish confidence intervals overlap more, but we maintain that Wordscores results are more valid and reliable
than Wordfish estimates.
In this chapter, we have limited our scope to two European languages that are
relatively close to each other. Including non-Indo-European languages in comparative research would be an interesting addition and a test for the universality of
computer-assisted content analysis. The inclusion of languages that have a different morphological structure, such as isolating languages (e.g. Chinese), with low
morpheme-per-word ratio, would be a good test of the ability of computer-assisted
content analysis to discriminate text positions based solely on word distribution.
Research in comparative linguistic is progressing fast (for example, see Yang and
Li 2003) and its applications could be useful for political textual analysis.
References
Benoit, K. and M. Laver. 2006. Party Politics in Modern Democracies. New York: Routledge.
Benoit, K. and M. Laver. 2007a. Benchmarks for text analysis: A response to Budge and
Pennings. Electoral Studies 26(1), pp. 130-135. DOI: 10.1016/j.electstud.2006.04.001
Benoit, K. and M. Laver. 2007b. Estimating party policy positions: Comparing expert surveys and hand-coded content analysis. Electoral Studies 26(1), pp. 90107. DOI: 10.1016/
j.electstud.2006.04.008
Benoit, K. and M. Laver. 2008. Compared to what? A comment on A robust transformation
procedure for interpreting political text by Martin and Vanberg. Political Analysis 16(1),
pp. 101111. DOI: 10.1093/pan/mpm020
Bruninger, T. and M. Debus. 2008. Der Einfluss von Koalitionsaussagen, programmatischen
Standpunkten und der Bundespolitik auf die Regierungsbildung in den deutschen Lndern.
Politische Vierteljahresschrift 49(2), pp. 309338. DOI: 10.1007/s11615-008-0101-6
Budge, I., H.-D. Klingemann, A. Volkens, J. Bara and E. Tanenbaum (eds). 2001. Mapping Policy
Preferences I. Estimates for Parties, Electors, and Governments 19451998. Oxford: Oxford
University Press.
Budge, I. 2000. Expert judgments of party policy positions: Uses and limitations in political research European Journal of Political Research 37(1), pp. 103113. DOI: 10.1111/
1475-6765.00506
Budge, I. and P. Pennings. 2007. Do they work? Validating computerized word frequency
estimates against policy series. Electoral Studies 26(1), pp. 121129. DOI: 10.1016/
Castles, F.G. and P. Mair. 1984. Left-right political scales: Some expert judgments. European
Journal of Political Research 12(1), pp. 7388. DOI: 10.1111/j.1475-6765.1984.tb00080.x
Debus, M. 2009. Pre-electoral commitments and government formation. Public Choice 138(1),
pp. 4564. DOI: 10.1007/s11127-008-9338-2
Huber, J.D. and R. Inglehart. 1995. Expert interpretations of party space and party locations in
42 societies. Party Politics 1(1), pp. 73111. DOI: 10.1177/1354068895001001004
Hug, S. and T. Schulz. 2007. Left-right positions of political parties in Switzerland. Party Politics
13(3), pp. 305-330. DOI: 10.1177/1354068807075938
Irvine, W. 1987. Canada 19451980: Party platforms and campaign strategies. In I. Budge,
D. Robertson and D. Hearl (eds), Ideology, Strategy, and Party Change: Spatial Analysis of
Post-War Elections Programmes in Nineteen Democracies. Cambridge: Cambridge University Press, pp. 7394. DOI: 10.1017/CBO9780511558771.005
Jian-Yun, N., M. Simard, P. Isabelle and R. Durand. 1999. Cross-language information retrieval
based on parallel texts and automatic mining of parallel texts from the web. In Proceedings
of the 22nd annual international ACM SIGIR conference on Research and development in
information retrieval. Berkeley, California, United States: ACM.
Johnston, R. 2008. Polarized pluralism in the Canadian party system: Presidential address to the
Canadian Political Science Association, June 5, 2008. Canadian Journal of Political Science/
Revue canadienne de science politique 41(4), pp. 815834.
Keman, H. 2007. Experts and manifestos: different sources Same results for comparative
research? Electoral Studies 26(1), pp. 7689.
Kleinnijenhuis, J., and P. Pennings. 1999. A probabilistic keyword approach to textual analysis.
In Mannheim Joint Sessions of the ECPR. Mannheim.
Klemmensen, R., S.B. Hobolt and M.E. Hansen. 2007. Estimating policy positions using political texts: An evaluation of the wordscores approach. Electoral Studies 26(4), pp. 746755.
DOI: 10.1016/j.electstud.2006.04.004
Klingemann, H.-D., A. Volkens, J. Bara, I. Budge and M.D. McDonald. 2007. Mapping Policy
Preferences II: Estimates for Parties, Electors and Governments in Central and Eastern
Europe: Oxford University Press.
Knutsen, O. 1998. The strength of the partisan component of left-right identity: A comparative
longitudinal study of left-right party polarization in eight west European countries. Party
Politics 4(1), pp. 531. DOI: 10.1177/1354068898004001001
Laver, M. 1998. Party policy in Britain 1997: Results from an expert survey. Political Studies
46(2), pp. 336347. DOI: 10.1111/1467-9248.00144
Laver, M. and K. Benoit. 2005. Estimating party policy positions: Japan in comparative context.
Japanese Journal of Political Science 6(2), pp. 187209. DOI: 10.1017/S1468109905001830
Laver, M., K. Benoit and J. Garry. 2003. Extracting policy positions from political texts using
words as data. The American Political Science Review 97(2), pp. 311331. DOI: 10.1017/
S0003055403000698
Laver, M., K. Benoit and N. Sauger. 2006. Policy competition in the 2002 French legislative
and presidential elections. European Journal of Political Research 45(4), pp. 667697. DOI:
10.1111/j.1475-6765.2006.00313.x
Laver, M. and J. Garry. 2000. Estimating policy positions from political texts. American Journal
of Political Science 44(3), pp. 619634. DOI: 10.2307/2669268
Laver, M. and W.B. Hunt. 1992. Policy and Party Competition. New York: Routledge.
Lederer, M. 1994. La traduction aujourdhui: le modle interprtatif. Vanves: Hachette F.L.E.
Lowe, W. 2008. Understanding wordscores. Political Analysis 16(4), pp. 356371. DOI: 10.1093/
pan/mpn004
Magin, R., M. Freitag and A. Vatter. 2009. Cleavage structures and voter alignments within
nations. Zeitschrift fr Vergleichende Politikwissenschaft 3(2), pp. 231256. DOI: 10.1007/
s12286-009-0062-1
43
44 Benot Collette and Franois Ptry
Mair, P. 2001. Searching for the positions of political actors: A review of approaches and a critical evaluation of expert surveys. In M. Laver (ed.), Estimating the Policy Positions of Political
Actors. New York: Routledge, pp. 1029.
Marks, G., L. Hooghe, M.R. Steenbergen and R. Bakker. 2007. Crossvalidating data on party
positioning on European integration. Electoral Studies 26(1), pp. 2338. DOI: 10.1016/
Martin, L.W. and G. Vanberg. 2008. A robust transformation procedure for interpreting political
text. Political Analysis 16(1), pp. 93100. DOI: 10.1093/pan/mpm010
McCallum, A. and K. Nigam. 1998. A comparison of event models for naive Bayes text classification. In AAAI-98 Workshop on Learning for Text Categorization.
Monroe, B.L., M.P. Colaresi and K.M. Quinn. 2008. Fightin words: Lexical feature selection
and evaluation for identifying the content of political conflict. Political Analysis 16(4),
pp. 372403. DOI: 10.1093/pan/mpn018
Proksch, S.-O. and J.B. Slapin. 2009. Position taking in European parliament speeches. British
Journal of Political Science 40, pp. 587611. DOI: 10.1017/50007133409990299
Ray, L. 2001. A natural sentence approach to the computer coding of party manifestos. In
M. Laver (ed.), Estimating the Positions of Political Actors. New York: Routledge, pp.149161.
Ray, L. 2007. Validity of measured party positions on European integration: assumptions,
approaches, and a comparison of alternative measures. Electoral Studies 26(1), pp. 1122.
DOI: 10.1016/j.electstud.2006.03.008
Slapin, J.B. and S.-O. Proksch. 2008. A scaling model for estimating time-series party positions from texts. American Journal of Political Science 52(3), pp. 705722. DOI: 10.1111/
j.1540-5907.2008.00338.x
Vinay, J.-P. and J. Darbelnet. 2003 [1977]. Stylistique compare du franais et de langlais :
Mthode de traduction. Laval: Groupe Beauchemin.
Volkens, A. 2007. Strengths and weaknesses of approaches to measuring policy positions of parties. Electoral Studies 26(1), pp. 108120. DOI: 10.1016/j.electstud.2006.04.003
Whitefield, S., M.A. Vachudova, M.R. Steenbergen, R. Rohrschneider, G. Marks, M., P. L
oveless,
and L. Hooghe. 2007. Do expert surveys produce consistent estimates of party stances
on European integration? Comparing expert surveys in the difficult case of Central and
Eastern Europe. Electoral Studies 26(1), pp. 5061. DOI: 10.1016/j.electstud.2006.04.006
Yang, Ch. C. and K. Wing Li. 2003. Automatic construction of English/Chinese parallel corpora.
Journal of the American Society for Information Science and Technology 54(8), pp. 730742.
DOI: 10.1002/asi.10261
Appendix
Document scores correlations
Wordscores
English
French
English (stemmed)
French (stemmed)
English
1.000
French
0.9710
1.000
English (stemmed)
0.9926
0.9509
1.000
French (stemmed)
0.9374
0.8983
0.9411
1.000
Wordfish
English
French
English (stemmed)
French (stemmed)
English
1.000
French
0.7020
1.000
English (stemmed)
0.9390
0.8009
1.000
French (stemmed)
0.9242
0.8154
0.9843
1.000
45
chapter 3
Leveraging textual sentiment analysis

withsocial network modeling
Sentiment analysis of political blogs
in the 2008 U.S. presidential election
Oxford Internet Institute, University of Oxford /

Oxford University Computing Laboratory
Automatic computational analysis of political texts poses major challenges for

state-of-the-art Sentiment Analysis and Natural Language Processing tools.
In this initial study, we investigate the feasibility of combining purely linguistic indicators of political sentiment with non-linguistic evidence gained from
concomitant social network analysis. The analysis draws on a corpus of 2.8 million political blog posts by 16,741 bloggers. We focus on modeling blogosphere
sentiment centered around Barack Obama during the 2008 U.S. presidential
election, and describe a series of initial sentiment classification experiments
on a data set of 700 crowd-sourced posts labeled for attitude with respect to
Obama. Our approach employs a hybrid machine-learning and logic-based
framework which operates along three distinct levels of analysis encompassing
standard shallow document classification, deep linguistic multi-entity sentiment
analysis and scoring and social network modeling. The initial results highlight
the inherent complexity of the classification task and point towards the positive
effects of learning features that exploit entity-level sentiment and social-network structure.
1. Introduction
Political blogs constitute a fascinating genre. They are typically charged with
extremely high amounts of personal opinions, sentiments, stances, feelings and
emotions towards and about a multitude of individuals, organisations, issues and
events that have some political relevance, and represent a very large number of
people. Due to the sheer size of such a complex distributed, dynamic and vibrant
information space, the only viable way of monitoring, analysing, and predicting
48 Wojciech Gryc and Karo Moilanen
information in the political blogosphere is to develop a large-scale computational

multi-paradigm and multi-modal framework that can deal with the content as well
as the social aspects of the blogosphere.
With this lofty goal in mind, we investigate in this study the feasibility of
combining purely textual indicators of political sentiment with non-textual information gained from concomitant social network analysis. We focus on political
blog analysis and base our research on a large corpus of posts by 16,741 bloggers,
crawled daily between April 2008 and May 2009. Rich political blog data allows
us to explore a number of major themes in political sentiment analysis, social network analysis, text mining, and, most importantly, how these areas interact. For
example, by exploring the content of each post and how it fits into the larger social
network in which it resides one can see that discussions tend to cluster differently
depending on individual politicians: people discussing Barack Obama tend to be
more dispersed than those discussing John McCain, for example. An interesting
and highly relevant research question is whether such network-based information
can be used to improve the accuracy of automatic political-sentiment classification.
In this initial study, we approach automatic sentiment analysis of political
blogs using a hybrid machine-learning and logic-based classification framework
which operates along three distinct complementary levels of analysis, each capable
of offering a unique representation of each blog post:
Shallow document classification (4.1): as a first strong baseline, centered
around holistic document-level textual evidence alone, each blog post is represented using standard n-gram features.
Deep entity-level sentiment scoring (4.2): in a second approach centered
around much more focused lower-level sentiment evidence, each blog post is
represented as a set of detailed sentiment scores assigned to all individual entities mentioned in the post (e.g., politicians, places, organisations, and abstract
issues).
Social network modeling (4.3): in a third, much wider meta-approach centered around the blog posts social context, each blog posts position in the
social network structure across the whole blog-post space is used as classification evidence.
Classification task
Although our framework is not restricted to any particular topic or entity (be they
concrete or abstract), we confine1 ourselves in this initial study to only one specific entity in order to gain a better understanding of the problem. In light of the
1. We aim to expand this in future to further politicians and topics such as McCain and Iraq.
Chapter 3. Leveraging textual sentiment analysis with social network modeling
importance of social media to Barack Obamas political campaign in the 2008 U.S.
presidential election, we decided to focus our analysis on the sentiment expressed
in the blogosphere towards and about Barack Obama within the time period that
our blog data represents. The classification task that we attempt in this initial study
can be summarised specifically in the following question:

Given a political blog post, does it, as a whole, express positive, neutral, or negative sentiment towards or about the target entity (Barack Obama)?
The classification task can be characterised accordingly as entity-centric documentlevel sentiment classification because the document-level sentiment polarity label
of a given post reflects the overall sentiment towards or about a single target entity
in that post, not the overall sentiment of the post as a whole. Note that this type
of sentiment classification is different from the traditional document-level classification paradigm because a post that is negative overall (i.e., as a document)
can (and is likely to) be concurrently negative, positive, or neutral towards or
about many individual entities. We approach the problem as if we were trying
to label the entire data set after it was collected, rather than in real time. We are
more interested in being able to label a majority of the 2.8 million posts we have
collected with some level of confidence rather than having to look into the future
when labeling specific posts.
General challenges
Automatic computational analysis and categorisation of political texts is on the
whole a seriously challenging task, as has been observed in the area (e.g., Durant
and Smith 2007; Malouf and Mullen 2008; Mullen and Malouf 2006; Thomas et
al. 2006; Yu et al. 2008). When the analytical scope is extended to include further
non-factual aspects of meaning pertaining to subjectivity, sentiment, opinions,
affect and emotions, the analytical and computational challenges become even
more pronounced. This is especially the case with politically charged content in
blogs because, as a genre, political blogs represent noisy, in-depth, collaborative,
and dynamic discussions and debates by multiple contributors across a wealth
of topics, issues and entities only some of which constitute core content while
some others are mere digressing or tangential content. Blog posts further link to
each other via highly complex interrelated direct/explicit and indirect/implicit
structural, semantic, rhetorical, and temporal chains. It can be argued that no such
chain can be explained fully out of context, although blog posts are likely to carry
at least some clues that an algorithm can exploit. In addition to their distributed
nature, blog posts can also include other forms of multimedia such as embedded
videos or images which existing Natural Language Processing (NLP) algorithms
cannot easily align with the text content.
49
50
From the viewpoint of textual sentiment-analysis algorithms, any classification evidence that may be gleaned from blog posts is inherently noisy because
bloggers real sentiments and opinions are often obfuscated by complex rhetoric,
irony, sarcasm, comparisons, speculation and other paralinguistic devices that
prototypically characterise political discussions. In addition, domain-specific
terms, word senses, and vocabularies and informal/non-standard registers also
feature frequently. As is the case with Web content in general, political blogs also
come with many purely textual hurdles that have to be faced by NLP tools such
as complex or incomplete grammatical structures, broken sentence boundaries,
quotes, junk characters and spelling anomalies.
Even if such structural and textual problems were solved fully, the very task
of automatically detecting bloggers political sentiments, opinions, and orientation pertaining to highly polarised political issues would still remain formidable.
This stems from the fact that current computational tools be they linguistic or
non-linguistic struggle to map raw surface clues onto deeper semantic representations which ultimately require, for each blogger and for each issue, bill, or event
under consideration, the ability (i) to detect the bloggers political party affiliation,
political viewpoints, professional background, motivation, general knowledge
and, indeed, affective states; (ii) to measure the bloggers political extremeness
or distance from a centrist position; (iii) to measure the bloggers confidence and
agreeability/argumentativeness in the discussions; (iv) to measure how politically
important something is to the blogger; (v) to understand how meaning was constructed collaboratively by the bloggers; (vi) to understand why certain topics
are (not) discussed; and (vii) to detect sincere opinions vs. deliberate flaming.
Moreover, not only are many political opinions latent behind expressions which
(from an algorithms point of view) present themselves as purely neutral but some
explicit political opinion expressions may even be inversely related to the bloggers
actual political orientation.
2. Data
2.1
Election data
The present initial study, which is part of a wider research project, focuses on data
provided by IBMs Predictive Modeling Group. In total, the data set consists of
2,782,356 posts written by 16,741 politically-oriented bloggers collected between
April 22nd 2008 and May 1st 2009 (many of which focus on the U.S. Presidential
election in 2008). The selection of blogs was based on the tags associated with
them on the Technorati2 blog indexing service. Such a blog data set provides an
interesting opportunity for researchers because it contains text-based discussions
on politics as well as date stamps and hyperlinks between blog posts. The hyperlink feature can be treated as a citation network which shows the various ways in
which individual bloggers are and become aware of each other, and how information flows within the political blogosphere.
Posts were found through blogs self-reporting new content through the RSS
(Real Simple Syndication) standard, which includes hyperlinks to the content
posted on the blogs themselves. Once the blogs were crawled, the content of each
post was filtered to discard text and hyperlinks from advertisements, side bar content (e.g., blog rolls) and other portions of the web pages that include unwanted
superfluous content.
The role of content filtering is extremely important, mainly for three reasons.
Firstly, blog titles and content from sidebars can easily bias any statistical models
trained on noisy data. Since we are only interested in the main content of every
blog post, it is important to be able to avoid such bias by discarding as many sidebar links as possible. Secondly, superfluous content can, even in structural terms,
cause potentially devastating complications for NLP tools which can manifest
themselves as incorrect sentence breaking, part-of-speech tags, phrase chunking,
and parsing all of which will deteriorate the performance of sentiment classifiers
that use NLP components. Lastly, blogs often acted in an automated fashion. For
example, blog A citing blog B can automatically cause blog A to leave a link as a
comment on blog Bs post and hence result in additional links. Blog rolls, while
providing useful social-network information, may cause the network to become
artificially dense with hyperlinks that do not pertain to the blog posts themselves.
It is important to note, however, that while our filtering strategy is effective enough
for practical purposes, it is based on heuristics and is by no means perfect.
2.2
Sentiment annotations
While the aforementioned data collection process allows us to analyse the content
of the posts and to build and model social networks of bloggers and their posts, the
crawling process did not as such deal with the sentiment expressed in the posts.
In order to gain access to the sentiment properties of the posts, we made use of
Amazons Mechanical Turk3 (MT) service which involves Requesters (us) posting
2. http://technorati.com/, last accessed March 6, 2014.
3. https://www.mturk.com/mturk/welcome, last accessed March 6, 2014.
51
52
batches of small tasks known as HITs (Human Intelligence Tasks) which are completed by Workers (Turkers) for a small monetary reward ($0.03 per blog post).
For the present initial study, we selected a random sample of 700 blog posts
discussing Barack Obama and asked the Turkers to label them as (i) positive,
negative, neutral towards or about Obama, or (ii) not applicable (with
respect to Obama). In order for us to monitor and ensure the post-level consistency of the Turkers sentiment ratings, we required each post to be labelled by
three Turkers. In total, 86 unique Turkers took part in the task. All posts labeled as
not applicable were discarded because such cases typically indicate that a given
blog post to be labeled had been taken offline or contained irrelevant non-textual
content (e.g., a video or an image). From the resulting raw sentiment ratings, two
labeled subsets were generated as follows:
Lenient majority vote: The first subset contains all posts that received a majority vote whereby either 2/3 or 3/3 Turkers had to agree on the label of each
post. This resulted in 454 posts from which a final4 labelled corpus of 439
posts was obtained. All results reported in the present initial study are from
this subset.
Strict agreement: The second subset contains all posts that received unanimous
3/3 votes. Since that criterion resulted in only 124 posts, this subset is not
included in the present study.
We are in the process of increasing the amount of sentiment annotations.
Human performance. Even though it is complicated to estimate the interannotator agreement rates between 86 annotators (each of whom provided
different amounts of annotations), some tentative observations can be made
regarding the expected human agreement and performance ceiling in the sentiment classification task that our classifiers aim at solving. Table 1 shows the
distribution of votes per sentiment polarity, taking 2/3 and 3/3 agreement
ratings and excluding not applicable cases. It interestingly reveals that only
126 (27.75%) display full agreement, 145 (31.94%) display disagreements
that involve neutral polarity (neutral disagreement) and 183 (40.31%)
display fatal disagreement in the form of opposing non-neutral polarities.
Both the noticeably low amount of full agreement and the relatively high
amount of fatal disagreement suggest that, perhaps not surprisingly, the
3-way classification task is highly subjective even for humans. It is against the
strict ceiling of 27.75% (or the lenient one without fatal disagreement cases
at 59.69%) that the classifiers performance ought to be compared.
4. A small amount of posts were excluded due to anomalous content and features.
Data quality. This kind of empirical-data crowd sourcing offers practical benefits that cannot be refuted. It can further be argued that, because they reflect
real, uncontrolled, and untrained opinions, crowd-sourced sentiment annotations are maximally valid in terms of their naturalness. The downside is
naturally that the quality of the resultant annotations may be lower than what
can be reached in traditional, more rigorous controlled and vetted annotation
campaigns. In particular, the very nature of the Mechanical Turk service is
based on the notion of quantity rather than quality as the Turkers expect simple tasks that do not require any comprehensive annotator training as such5
so that each can be completed in a matter of seconds to reflect the typically
paltry per-item pay rates.
Despite these quality concerns, the use of crowd sourcing as a data collection
method has proven effective for machine learning in general and sentiment analysis in particular (Hsueh et al. 2009).
Table 1. Distribution of 3-way sentiment judgements from 86 annotators.
POS
NTR
NEG
NTR (3)
NTR (2)
46
45
34
1
10.13%
9.91%
7.49%
0.22%
POS (1) NTR (2)

NTR (2)
NTR (1)
POS (2) NTR (1)
NEG (1)
NEG (2)
43
40
39
23
9.47%
8.81%
8.59%
5.07%
POS (1)
POS (2)
POS (1) NTR (1)
POS (1)
NEG (2)
NEG (1)
NEG (1)
NEG (1)
82
56
44
1
18.06%
12.33%
9.69%
0.22%
NEG (3)
POS (3)
Agreement type
FULL AGREEMENT
FULL AGREEMENT
FULL AGREEMENT
FULL AGREEMENT
126 27.75%
NEUTRAL DISAGREEMENT
145 31.94%
FATAL DISAGREEMENT
FATAL DISAGREEMENT
FATAL DISAGREEMENT
FATAL DISAGREEMENT
183 40.31%
454 100.00%
Hsueh et al. (2009), for example, carried out an analysis of sample-post snippets
from the same pool of blog data as ours and confirmed the high quality of ratings
from Turkers against those from expert annotators. Turker annotations have also
been used to rate Wikipedia articles (Kittur et al. 2008) and news headlines (Snow
5. A typical HIT page contains only simple instructions and/or examples.
53
54
et al. 2008): in both studies, the quality6 of Turker ratings was also comparable to
that of expert annotators.
We hence conclude that the seemingly low inter-annotator agreement rates
on our data set were not the byproduct of crowd-sourced annotations as such
but rather reflect the inherently fuzzy and subjective properties of the underlying
sentiment classification task.
3. Related work
Due to the fact that our classification framework involves a complex network of
tools, ideas, and phenomena from multiple paradigms, topics, areas, and fields,
we cannot provide a full survey here. We hence limit the discussion on the relation of the present study to past work and existing proposals to political sentiment
analysis.
The recent surge of interest in mainstream (e.g., product/movie revieworiented) Sentiment Analysis and Opinion Mining (Pang and Lee 2008) has
touched upon the political domain in the form of a few studies which have followed the standard document topic-classification paradigm and which have simply
mapped the default positive-neutral-negative sentiment polarities onto political
polarities in bi- or tripartite political systems. These document-level approaches
typically use some form of machine learning with no or only shallow linguistic
features. Mullen and Malouf (2006) discuss the application of sentiment analysis
to informal political discourse to predict political affiliations as right (Republican,
conservative, r-fringe) vs. left (Democrat, liberal, l-fringe) in blog posts using a
probabilistic classifier (accuracy ~60.37%). In a similar study, Malouf and Mullen
(2006) used web-based Pointwise Mutual Information scoring, supervised machine
learning, and citation graph clustering (accuracy 68.48%, ~73%).
Other variants of the same paradigm have focused on classifying public comments on proposed governmental regulations as pro vs. against with a combination of sentiment analysis, (sub)topic detection, argument structure analysis, and
semantic frame analysis (Kwon et al. 2006). The cultural orientation and ideologies
in left- and right-wing political documents were estimated based on co-citation
information in Efron (2004) (accuracy ~90%) while congressional floor debates
were classified as support for vs. opposition to to (a piece of legislation) using
6. Anecdotal evidence and general opinions amongst the users of crowd-sourced annotations
however suggest that their quality depends (sometimes entirely randomly) on the annotation
task attempted and the pool of Turkers that participated in it, and that the risk of obtaining junk
annotations is always present.
graph-based agreement links between speech segments and speaker identities in

Thomas et al. (2006) (accuracy ~70.81%). Others have involved various forms of
supervised learning to classify congressional floor debates for general sentiment
(Yu et al. 2008) (accuracy ~65.5%); to capture what kinds of subjective perspectives (points of view) are expressed in text pertaining to the israeli vs. palestinian polar classes (Lin et al. 2006) (accuracy 93.46~99.09% for documents,
~94.93% for sentences); and to classify left-voice vs. right-voice blog posts
about President Bushs management of the Iraq War (Durant and Smith 2007)
(accuracy ~89.77%), amongst others.
Similar approaches can be found in the form of predictive models which
include temporal features. An opinion-forecasting approach was described in
Lerman et al. (2008) who combine a shallow bag-of-words approach with predefined entities, syntactic parsing, and temporal news coverage models to predict
the impact of news on public perception of political candidates. Kim and Hovy
(2007) in turn describe a supervised learning system that predicts which party is
going to win the election on the basis of opinions posted on an election prediction
website (accuracy ~81.68%).
While these studies have reported relatively high accuracy levels, the standard document topic classification paradigm is too coarse to score individual entities. To the best of our knowledge, the role of deeper sentiment analysis in general
and fine-grained multi-entity scores in particular has not been investigated fully
in the area of political sentiment analysis. The approach closest to ours in this analytical vein, centered around entities, is Van Atteveldt et al. (2008b) who describe
a system for automatic analysis of Dutch political newspaper texts that exploits
basic semantic sentiment roles such as opinion sources in constructing a basic
conceptual semantic network representing a given document. The authors extract
semantic relations between political actors derived from syntactic parsing, rulebased mappings between syntactic and semantic roles, political ontologies, a basic
anaphora resolution mechanism, and template-based pattern matching to find
opinion source constructions and to determine the general semantic agent and
patient roles. In the task of determining basic semantic roles at the level of documents, the authors reached 53% (precision): 31% (recall) for opinion sources,
57% (precision): 51% (recall) for agents, and 39% (precision): 36% (recall) for
patients. Instead of classifying entities as such, a related study in Van Atteveldt et
al. (2008b) attempted to classify specific positive/neutral/negative relations (e.g.,
support/criticism, success/failure, evaluation) between political actors and issues
in the news coverage of the Dutch 2006 elections. The authors focused on relevant salient words subsumed by subject and object nodes in dependency parse
trees as features for machine learning (alongside part-of-speech tag, n-gram, and
semantic thesaurus class information). An overall F-score of ~63% was reported.
55
56
The main difference between these methods and our proposed approach is that
the former afford only partial coverage of some pre-specified entities of interest in
pre-specified syntactic and semantic positions while a more comprehensive multientity scoring framework captures all individual entity mentions and hence offers
considerably higher recall.
4. Overview of the classification framework
4.1
Shallow document classification
One of the simplest approaches to text categorisation is arguably the bag-of-words

paradigm in which a given text is represented as an unordered collection of independent statistical features pertaining to word or n-gram frequencies. Although
structural and positional information about words and n-grams is discarded altogether, the bag-of-words method is hard to beat in practice. We therefore adopt
as a strong baseline classifier a unigram count model (Manning et al. 2008) that
operates on stemmed7 and normalised n-grams from the blog posts. Rather than
using raw word frequencies directly, we use TF-IDF (Term Frequency-Inverse
Document Frequency) vectors to represent the posts themselves. We limited
the word vectors to terms that appear in at least 5 different posts, and avoided
terms that appeared in over 500 posts. This resulted in 4647 unigram features for
classification.
4.2
Deep entity-level sentiment scoring
In order to complement deeper linguistic and sentiment information the holistic

evidence offered by the shallow n-gram method, we employed a wide-coverage
sentiment parser. In particular, we wished to utilise in the analysis information about the overall sentiment expressed in each blog post towards all individual entities mentioned in it. The parser, which is described in greater detail
in Moilanen and Pulman (2009), employs compositional sentiment logic, deep
grammatical analysis, and large manually compiled sentiment lexica to exhaustively assign sentiment scores to different structural levels across individual
words, syntactic phrases, sentences, and documents. In particular, it assigns gradient POS:NTR:NEG sentiment scores for all individual entity mentions (e.g.
Obama(+), Obamas(), Barack(N), Chicago(+), ) and aggregated entity
7. As output by the standard Porter stemmer (Porter 1980).
In order to complement with deeper linguisticTriangle

and sentiment
information
the holistic
evidence
ofX
Y
POS
NTR
NEG
Tuesdaya wide-coverage
41.000
45.00
48
fered by the shallow n-gram method, we employed
sentiment
parser.
In45particular,7
night
44.000
38.00
53
38
9
we wished to utilise in the analysis informationClintons
about the overall
expressed
in24each blog9
58.000 sentiment
24.00
67
campaign
58.000
0.00
79
0
21
individual
entities mentioned
inspotlight
it. The parser,
which
issocial
described
in
26.000
0.00
63 in greater
0 detail 37
post towards all
Chapter
3. Leveraging
textual sentiment
analysis
with20.00
network
modeling
57
which grammatical
-28.000 analysis,
26
20
54
[19], employs compositional sentiment logic, deep
and
large
manually
comrisk
-50.000
0.00
25
0
75
Obama
-62.000 to different
16.00
11
16
73
piled sentiment lexica to exhaustively assign sentiment
scores
structural
levels
across
individual words, syntactic phrases, sentences, and documents. In particular, it assigns gradient
(+) , Obamas(-) ,
POS : NTR
:NEG
sentimenthad
scores
for all individual
entity
mentions
(e.g. Obama
topics
(e.g.
Obama
25 mentions
58% of
which
were positive)
in a given
blog
(N)
(
+
)
Barack , Chicago , ...) and aggregated entity topics (e.g. Obama had 25 mentions 58% of
post.
Example
(1)
shows
a
sample
sentence
from
our
corpus
and
the
gradient
which were positive) in a given blog post. Ex. 1 shows a sample sentence from our corpus and the
it: it.
gradient sentiment
that
the parser
assigned
to the
[ ENTITIES ] inin
sentiment
scores scores
that the
parser
assigned
to the
[ENTITIES]
Judging by [T UESDAY ] [ NIGHT ], the
(1)
Judging by [TUESDAY] [NIGHT],
[C LINTONS ] would want to share the

[CLINTONS
] would],want
to ]
[the
CAMPAIGN
] [ SPOTLIGHT
[ WHICH
share the
the [CAMPAIGN
]
Mr.
runs
[ RISK ] of making
(1)
[O
BAMA ] look weak.
[SPOTLIGHT
], [WHICH]
runs the [RISK] of making

Mr. [OBAMA] look weak.
<=>#
("!#
!"'#
!"&#
=CD0E5F#
!"%#
34565#
!"$#
-,12.#
92,:2#
0;/.+,12.#
7,08#
*+,-./-0#
:56;5,1-#
!"!#
)(#
)!"'#
)!"&#
)!"%#
)!"$#
!#
!"$#
!"%#
!"&#
!"'#
(#
<AB#
?3@#
transformedthe
the aggregated
aggregated topical
POS
:NTR:NEG sentiment
percentage
counts from all
We We
transformed
topical
POS:NTR:NEG
sentiment
percentage
entities mentioned across all 700 posts into unique learning features (e.g. OBAMA NEG SCORE,
counts
from all entities mentioned across all 700 posts into unique learning
PALIN NTR SCORE , IRAQ NEG SCORE , ...). Ex. 2 shows the top 25 most frequent topical entities
features
(e.g., OBAMA_NEG_SCORE, PALIN_NTR_SCORE, IRAQ_NEG_
from the posts.
SCORE,).
Example (2) shows the top 25 most frequent topical entities from
(2)
Obama (115), that (97), you (91), he (91), it (91), barack (88), I (84), they (81), we (81), who
the posts.
(79), what (70), McCain (69), people (67), this (65), campaign (64), candidate (59), there
(56), president (53), John (52), time (51), election (51), all (49), one (48), comment (48), day
(2)
Obama (115), that (97), you (91), he (91), it (91), Barack (88), I (84), they (81),
(46)
we (81), who (79), what (70), McCain (69), people (67), this (65), campaign
In order to make the features more focused around the key entities and issues in the election,
(64), features
candidate
(59),
there
(56),
(53),
Johnacross
(52), our
timecorpus.
(51), election
we discarded
from
entities
that
had president
a frequency
of <10
This filtering
resulted (51),
in 951all
entity
score
features
for
classification.
(49), one (48), comment (48), day (46)
In
to make
the features
more focused around the key entities and issues in
4.3order
Social
Network
Modelling
the
election,
we
discarded
features
from entities that had a frequency of <10 across
In addition to serving as an interesting text corpus of political sentiment, our data set contains rich
our
corpus.
This
filtering
resulted
in
951
entity bloggers
score features
for classification.
information about the relationships between
individual
represented
by various hyperlinking
structures. As many bloggers link to each other within their posts, such link data can reveal the rich
underlying social structures in the political blogosphere. Past research into such linking patterns
duringSocial
the 2004
U.S. presidential
election has, for example, shown that bloggers tend to segregate
network
modeling
4.3
themselves on ideological grounds, with conservative and liberal bloggers separating into tight-knit
clusters that have different behavioural characteristics with regards to hyperlinking to other blogs
In
addition
to servinghas
as an
interesting
text in
corpus
of political
sentiment,
our datasuch
([1]).
Such segregation
also
been observed
more topically
focused
blog communities
as
war
blogs
([25]).
Analysis
of
links
between
ideologically-charged
blog
clusters
similarly
shows
set contains rich information about the relationships between individual bloggers
that links are often used to critique other bloggers ([8]).
represented by various hyperlinking structures. As many bloggers link to each
Since blogs often self-categorise into ideologically-charged clusters, incorporating information
other
their posts,
can reveal
the rich
underlying
strucabout within
such clusters
into oursuch
blog link
post data
categorisation
model
appears
intuitively social
beneficial.
We accordingly
sought
to
investigate
the
possibility
of
leveraging
the
above
kinds
of
social
linking and
tures in the political blogosphere. Past research into such linking patterns during
sorting phenomena observed in the political blogosphere to facilitate the blog post classification
the
2004 U.S. presidential election has, for example, shown that bloggers tend to
and labelling task. In this initial study, we take a relatively simple approach to exploring which
segregate
themselves
on ideological
conservative
liberal
blogclusters bloggers
find themselves
in, with grounds,
posts then with
acquiring
the featuresand
of their
parent
blogs.
Weakly
Connected
Components.
While
one
could
look
at
the
individual
post-level
linking
gers separating into tight-knit clusters that have different behavioural characteris- patterns between blogs (e.g. [14]), the small number of post labels to which we currently have access
tics with regards to hyperlinking to other blogs (Adamic and Glance 2005). Such
segregation has also been observed in more
7 topically focused blog communities
such as war blogs (Tremayne et al. 2006). Analysis of links between ideologicallycharged blog clusters similarly shows that links are often used to critique other
bloggers (Hargittai et al. 2008).
58
Since blogs often self-categorise into ideologically-charged clusters, incorporating information about such clusters into our blog-post categorisation model
appears intuitively beneficial. We accordingly sought to investigate the possibility
of leveraging the above kinds of social linking and sorting phenomena observed
in the political blogosphere to facilitate the blog-post classification and labeling
task. In this initial study, we take a relatively simple approach to exploring which
clusters bloggers find themselves in, with posts then acquiring the features of their
parent blogs.
Weakly connected components. While one could look at the individual post-level
linking patterns between blogs (e.g., Leskovec et al. 2007), the limited number of
post labels to which we currently have access combined with sparse post-level
networks means that not enough post-level social information may be gleaned
from the data. Social networks at the blog level do however provide more information as all hyperlinks observed between blogs in the data set can be aggregated
into a directed network. We treated the connections between blogs as unweighted:
in particular, if at any point during the year blog A was linked to blog B, then we
treated this as a link in the blog graph. The one shortcoming of such an approach
is that relationships between bloggers that regularly link to each other are treated
the same as one-off links.
Using the iGraph package8 (Czrdi and Nepusz 2006), we determined the
location of each blog in the social network, alongside a number of different postspecific properties. In each case, we explored different subgraph types and whether
posts were written by a blogger situated in specific subgraphs of the network. For
example, a simple subgraph structure within a directed network is a weakly connected component which represents subgraphs where all nodes are connected to all
other nodes. In our blog network, the largest such weakly-connected component
consists of 8,297 blogs, while the second largest has 67. In this case, two variables
could be added to the feature vector used to classify a given blog post, namely
(1)is the parent blog situated in the largest component, and (2) is the parent blog
situated in the second-largest component? From these variables, ten boolean features were generated that indicate which community a given post belongs to, based
on which blog was responsible for posting it.
Community detection algorithms. A second plausible approach is afforded by
community detection algorithms which aim at finding dense subgraphs within
a social network. While a component can be one interpretation of a community
within a social network, community-finding algorithms often have more stringent
8. http://igraph.sourceforge.net/, last accessed March 6, 2014.
definitions. The general idea behind many such algorithms is to locate parts of a
social network that are more dense than one would expect based on the network as
a whole. Such a dense subgraph may imply stronger relationships between members of the subgraph compared to members outside of the subgraph. If we assume
that the social network is homophilous by ideology (McPherson et al. 2001) that
is, if blogs tend to link to each other more often when they share similar political
views then we can use their membership within subgraphs as features for classification. Using a fast greedy community detection algorithm (Clauset and Newman
2004), a number of communities were detected in the blog network. Fast greedy
community detection places nodes into communities in a way that maximises the
number of edges within communities, rather than between them. It is an agglomerative approach where each node begins by being within its own community, and
communities are then merged to maximise within-community links and minimise between-community links. The ten largest communities found9 using this
approach are outlined in Table 2.
Table 2. The ranks and sizes of the largest communities found
in the aggregated blog network.
Rank
10
Size
2077
1659
1131
945
582
484
380
355
98
82
From the above approaches to modeling the social-network structure of our blog
sample, 22 cluster features were generated for classification. These come in three
major categories:
1. Two weakly-connected components. Two of the features track whether the specific blog belongs to one of two of the largest weakly-connected components
in the network.
2. Ten communities (1). Membership within ten of the largest communities as
determined by the fast-greedy community detection algorithm is tracked as
the next set of features.
3. Ten communities (2). Membership within ten of the largest communities as
determined by the leading eigenvector approach to community detection. This
is similar to the earlier feature set, but it should be noted that different community detection algorithms (indeed, different random seeds) provide different membership categories.
9. Note that the blog network was symmetrised prior to running the fast greedy algorithm.
59
4.4 Overview of algorithms

For all experiments, we used standard Nave Bayes Multinomial (NBM) and
Logistic Regression models available in the WEKA toolkit (Hall et al. 2009), all
with their default parameters. By assuming that individual terms appear in a blog
post independently of all other terms, the NBM classifier calculates the probability
of a given blog post belonging to a positive, neutral, or negative category. This is
done by summing up the estimated log-probabilities of individual terms appearing in the categories.
We further made use of 2nd-tier Logistic Regression metaclassifiers which
use as their inputs the probabilistic predictions made by three 1st-tier NBM
classifiers. As metaclassifiers, we compared two different options, namely (1) a
Majority Voting classifier which counts the three class labels predicted by the 1sttier classifiers and assigns the majority class label to a given blog post, and (2) a
Stacking classifier which takes three inputs from the 1st-tier classifiers and treats
them as features for the final classification step.
We further experimented with Support Vector Machines (SVMs) and J48
decision tree algorithms. Because they did not perform as well as the above classifiers on our data set, their results are not included in this paper.
5. Experiments
5.1
Experimental conditions
We report the performance of three different feature types (4) across 5620 features in the following conditions:
1.
2.
3.
4.
5.
A Logistic Regression classifier with 22 social network features [SNA]

An NBM classifier with 951 sentiment analysis features [SA]
An NBM classifier with 4647 unigram bag-of-words features [BOW]
An NBM classifier with all 5620 features [ALL]
A Stacking classifier with three separate NBM classifiers for (1), (2), and (3)
[STACK]
6. A Voting classifier with three separate NBM classifiers for (1), (2), and (3)
[VOTE]
Each condition was measured through 10-fold cross-validation which splits the
data set into ten different folds (9/10 training vs. 1/10 testing). Each cross-validation run was further seeded with ten different seeds. All reported scores (unless
stated otherwise) represent averages from the 1010-fold cross-validation runs.

Three separate baselines are given to reflect a classifier that always outputs a given
polarity. The results from a three-way (POS vs. NTR vs. NEG) classification condition are given in Table 3 (below).
Table 3. Average 3-way 10-fold cross-validation results.
BASELINE ACCURACY (RAW)
POS
NTR
NEG
27.56
41.69
30.75
SNA
(22)
SA
(951)
BOW
(4647)
ALL
(5620)
STACK
(5620)
VOTE
(5620)
ACCURACY (RAW)
41.05
46.33
50.34
49.70
49.29
49.27
ACCURACY (PAIRWISE)
ACCURACY (POS)
ACCURACY (NTR)
ACCURACY (NEG)
60.70
68.20
48.79
65.10
64.22
65.88
55.76
71.03
66.89
68.06
58.66
73.96
66.47
69.61
57.08
72.71
66.20
70.59
54.83
73.17
66.18
69.20
56.63
72.71
PRECISION
PRECISION (POS)
PRECISION (NTR)
PRECISION (NEG)
39.53
37.34
42.69
38.57
45.85
36.32
47.54
53.68
50.52
40.72
50.32
60.52
49.40
42.87
48.86
56.48
49.22
42.08
47.31
58.26
49.43
41.92
48.54
57.81
RECALL
RECALL (POS)
RECALL (NTR)
RECALL (NEG)
37.38
22.64
66.67
22.81
44.35
31.65
58.74
42.67
48.01
34.46
65.41
44.15
47.49
30.58
62.79
49.11
45.33
17.52
73.50
44.96
46.47
30.33
67.38
41.70
F-SCORE
F-SCORE (POS)
F-SCORE (NTR)
F-SCORE (NEG)
36.30
28.18
52.05
28.66
44.63
33.82
52.54
47.53
48.41
37.32
56.88
51.04
47.72
35.67
54.95
52.53
44.33
24.70
57.56
50.73
46.68
35.18
56.43
48.45
SAR
50.43
54.28
56.87
56.46
55.81
56.04
KAPPA
8.40
19.62
25.15
25.52
22.55
23.38
KRIPPENDORFF
44.56
52.84
54.51
55.64
51.38
53.25
PEARSON
12.14
23.35
28.67
30.81
28.74
27.80
SPEARMAN
12.12
23.46
28.74
31.18
29.37
27.99
FATAL ERRORS
13.15
17.55
16.73
14.69
10.92
14.51
GREEDY ERRORS
23.56
32.05
29.03
30.83
21.78
26.79
LAZY ERRORS
63.29
50.39
54.24
54.48
67.30
58.70
61
62
5.2
Evaluation measures
A large number of different evaluation measures can be used to characterise the

performance of the classifiers and the features used, each of which highlights a
different evaluative aspect.
Accuracy. The first measure set targets the standard notion of accuracy used in
traditional factual classification tasks encompassing Accuracy, Precision, Recall,
F-Score, and SAR measures. For these, individual pairwise polarity conditions
(POS vs. NOT-POS, NTR vs. NOT-NTR, NEG vs. NOT-NEG) were used. In addition, raw percentage accuracies are reported. Although sentiment interpretation
can not be said to be (in)accurate in the strictest sense of the term, these measures
characterise the overall behaviour of our classifiers in a useful way.
Agreement. The second set of measures focuses on different levels of agreement
and correlation between human sentiment judgements and our classifiers by calculating chance-corrected ternary (POS vs. NTR vs. NEG) rates based on the
standard Kappa k, Pearsons r product moment correlation coefficient, Spearmans
rank order correlation coefficient, and Krippendorf s reliability coefficient
measures.10
Error types. The inter-annotator agreement levels point towards increased ambiguity with NTR polarity due to differing personal degrees of sensitivity towards
neutrality/objectivity. Not all classification errors are then equal for classifying a
POS case as NTR is more tolerable than classifying it as NEG, for example. We
found it useful to characterise three distinct error classes or disagreements between
human H and algorithm A. FATAL errors (H()A() {+ }) are those where the
non-neutral polarity is completely wrong: such errors affect the performance of a
classifier adversely. GREEDY errors (H(N)A() {+ }) are those where the algorithm wrongly made a decision to jump one way or the other, displaying oversensitivity towards non-neutral polarities. LAZY errors (H()A(N) {+ }) indicate
that the algorithm chose to sit on the fence and displayed oversensitivity towards
NTR polarity. We naturally aim at minimising FATAL errors.
10. All accuracy and agreement measures were obtained using R (http://www.r-project.org/)
with The built-in correlation functions together with the ROCR (http://cran.r-project.org/web/
packages/ROCR/) and IRR (http://cran.r-project.org/web/packages/irr) packages. All pages
were last accessed on March 6, 2014.
5.3
Discussion
In absolute terms, the scores are modest. However, when compared to the low
human ceiling (27.75~59.69%) (2.2), they do in fact appear promising. In general, all classifiers easily surpassed the low non-neutral (27.6~30.75%) and neutral11 (41.69%) baselines. The standalone performance of the social network (SNA)
features was not as effective as we expected. In the light of the very small number
of features used (only 22), it is in fact surprising that the SNA features worked at
all. The average F-score obtained by the SNA features was low (36.3%) mainly due
to low recall for non-neutral sentiment. Their pairwise accuracy rates are more
favourable as they show that the SNA features are not making random non-neutral
predictions. When larger networks are incorporated in the future, social network
features can be expected to offer important supporting evidence in the classification task that we are attempting.
Equally promising is the performance of the sentiment analysis (SA) features
especially considering that they only reflect the sentiment scores of a handful of
entities and constitute a relatively small feature set (only 951). The underlying
compositional sentiment parser, from which the SA features stem, is very sensitive towards non-neutral sentiment which resulted in slightly higher FATAL and
GREEDY error rates for the SA features (cf. the lowest LAZY error rate). The SA
features are on the whole more balanced than the SNA ones.
Considering the much larger feature set, the performance of the holistic bagof-words (BOW) features was (perhaps unsurprisingly) very strong. The figures
from 5620 features suggest that, as features, unigram evidence still reflects the
sentiment properties of a blog post more closely than non-lexical evidence, even
when it comes to measuring sentiment towards or about a single entity. The BOW
features were behaviourally closer to the SA features than to the SNA ones which
may be due to the fact that the SA entity features latently represent salient unigrams (e.g., Obama).
Our hypothesis concerning the leveraging power of the SNA and SA features
was confirmed partially as small gains over the BOW model were obtained in
some conditions by using the entire feature set (ALL) for a single classifier. This
can in particular be seen in the higher agreement/correlation rates and lower
amounts of FATAL errors. Stacking and voting amongst the three individual NBM
classifiers provided further boosts in some conditions.
11. With the exception of the SNA features.
63
Interestingly, all classifiers displayed a general tendency towards faring worse

with positive polarity than with negative polarity, especially in precision and
recall. All classifiers reached their highest recall and F-scores with neutral polarity. Moreover, the amount of FATAL errors stayed below 17% throughout which
suggests that all feature types are generally pointing at the right direction. This
correlates with the fact that a full 31.94% of the annotations involved conflicting
annotations around neutral polarity (neutral disagreement). The classifiers
can accordingly be expected cope better with non-neutral sentiment. When all
neutral cases were excluded from the evaluation, a small subset of 90145 nonneutral test cases was examined in order to verify this. Although the subset is too
small to draw any definite conclusions, it can nevertheless shed light on how the
classifiers did actually do with the core non-neutral cases which are arguably more
important for a sentiment classifier than neutral, objective cases.
Table 4 confirms the expected complications caused by neutral polarity in that
non-neutral precision scores go as high as 75.68%. Interestingly, the boost over
the BOW features given by the non-lexical SNA and SA features is much clearer
in the 2-way condition as the combined ALL features and the STACK and VOTE
classifiers all outperformed the BOW classifier. These 2-way scores seem to confirm that the SNA and SA features can indeed be used to leverage shallow unigram
features in non-neutral cases.
Lastly, we looked at the most informative features based on the Chi-squared
and information gain measures across all 5,620 features. Of the top 50 features
ranked by the two measures, 29 were entity scores from the SA feature set, with the
negative entity score for Obama, Chicago, and the positive score for rhetoric
topping the ranks alongside high-ranking unigrams such as Wright, pastor,
and Rezko, amongst others. This further confirms the utility of the entity-level
SA features for our classification task. The SNA features did not rank high as features, however.
Table 4. Average 2-way 10-fold cross-validation results.
ACCURACY (PAIRWISE)
PRECISION
RECALL
F-SCORE
SAR
KAPPA
PEARSON
SNA
(22)
SA
(951)
BOW
(4647)
ALL
(5620)
STACK
(5620)
VOTE
(5620)
63.16
63.13
63.10
63.07
55.20
26.19
26.23
69.84
69.14
69.73
69.21
61.58
38.62
38.87
73.55
73.06
73.50
73.10
65.24
46.35
46.55
76.14
75.52
74.66
74.95
67.32
49.99
50.17
77.13
75.68
72.51
73.42
67.29
47.25
48.08
74.23
73.51
73.78
73.57
65.77
47.19
47.28
5.4
Significance of results
What is particularly interesting about the results above is the implied relationship between political ideologies and the social structure of the political blogs
themselves. Not only do these results imply that one can improve political text
mining through structural features of the system being analysed, but these results
also imply a relationship between social structure and political discourse within
the social system itself. Indeed, the results imply that simply knowing the social
structure (i.e. link relationships between bloggers) is enough to surmise the general political stance of many bloggers.
This is particularly important for research into political discourse, as the 2004
and 2008 elections experienced a large amount of political blogging, and social
media as a whole played a key role in U.S. political discourse during this time.
Since 2008 (when the data was collected), new forms of social media (e.g., Twitter,
Pinterest, etc.) have begun to pervade the political social media landscape. As
these social technologies become engrained in our society and as such, into
popular political discourse it is important to understand the interplay between
the social structure of those participating, and the results of political discourse
taking place within these communities. If the social structure alone can effectively
predict the political views of those participating in the discourse, we must strive to
understand the social and political reasons behind these results.
We encourage researchers computational linguists, machine learning
experts, and political scientists to take these results and explore whether similar
relationships between structure and discourse appear in other types of communities and social systems (be they online or not). While the future work we propose
in Section 5.5 focuses tactically on extending the classification accuracy of our
models, the findings also suggest broader applications to the fields of political science and opinion research.
5.5
Future work
The proposed classification framework and the results obtained in this initial study
open up many avenues for future work.
Sentiment analysis. Although their utility is intuitively appealing, it is unclear how
far the capabilities of current deep-sentiment analysis tools could bring the analysis considering the subjective nature of the classification task. The most obvious
avenue for improvement is to incorporate semantic sentiment-role information
(cf. Van Atteveldt et al. 2008a) alongside information about basic Named Entities
65
in order to have access to features expressing people, places, and organisations

but also to richer political domain knowledge in the form of ontologies and other
evidence. Regarding the deep general-purpose sentiment parser that we employed,
further improvements can arguably be made by tuning its underlying lexicon
towards the political domain. The entity-level sentiment scores that were used in
this study can further be boosted by resolving pronoun mentions to their antecedents (cf. Van Atteveldt et al. 2008a) which, judged by the number of pronouns
amongst the top-scoring topical entities in Example (2), can be expected to have
a tangible impact on the classifiers.
Another useful research question is whether longer bi- and tri-grams could
improve accuracy beyond what we would expect from further blog corpora.
Although longer n-grams can rudimentarily model some further linguistic features of political discourse, past research in document-level sentiment analysis
suggests that simple binary unigram presence features suffice.
Clustering by topic. Another approach to analysing political blogs comes in the
form of blending social-network analysis with entity extraction. For every post
in our corpus, we have a list of entities that are mentioned in the post, together
with their sentiment labels. It is then possible to extract all individual entities
mentioned in each post and build a bipartite network representing the data set.
In this case, we have a matrix with columns representing entities and rows representing the individual posts (e.g., a value of 1 at the ith row and jth column of the
matrix representation of the network denotes that the ith post mentioned the jth
entity). That representation would allow us to observe topic overlaps between different posts which would be analysed by constructing a social network between
posts, where an edge between posts exists if the posts have some amount (e.g.,
10) of entities in common with each other. Such a inter-document network can
help elucidate clusters and communities based on topics not wholly unlike how
we use community detection algorithms as part of the social network analysis
to locate groups of bloggers that tend to communicate with each other. In this
case, however, sharing large numbers of topics or entities in discussions may
mean that specific topics or at the very least, having topics in common lead
to similar sentiment scores. Unfortunately, carrying out such an analysis at this
stage yields very poor results for topic clusters: it appears that most blog posts
have a large number of entities in common which causes standard community
finding algorithms to group the entire set of posts together as one large community. We conjecture that this is an artifact of the selection method used for blog
posts discussing Barack Obama. At this stage, the topical clustering method still
merits further research.
Advanced social network analysis. At this stage, only a few algorithms have been
investigated to combine the various feature sets. While we could explore richer
features based on entity extraction, n-grams, and deeper social-network analysis discussed above, we would still be developing feature vectors for each case
and build classifiers that operate on them. Since the algorithms in Weka are not
optimised as such for dealing with social network analysis, one extension to our
research is to build algorithms that directly integrate predictions into a social
network, rather than using features that reflect the current membership-withincommunity approach. One can, in particular, use the outputs of the entity-level
sentiment scores or unigram probability estimates and apply them as labels in the
social network. The social network itself could then be used to reinforce the labels
and see which ones appear realistic based on previous observations.
Dataset. A major challenge with the current initial study was data sparsity. Only
700 posts out of 2.8 million were labelled, making it very difficult to extract useful
information from the social network features in the data set. We hypothesise that,
as more posts are labelled and one gets a finer-grained picture of how bloggers
self-organise themselves into topical and political communities, social network
features should become more relevant and important. Another issue that ought
to be investigated is the concept of data-set shift. The fact that many terms and
phrases in the political domain develop and change their (non-)affective connotations over time is a key challenge for any text-based political blog and sentiment
analysis framework (especially for shallow classification methods). For example,
a term such as Alaska may have had different affective connotations before and
after Sarah Palin was announced as the Republican vice presidential candidate.
Opinions and ideology. The experiments in this paper focus on extracting sentiment toward a specific politician or entity. A more difficult challenge would be
extracting the ideology of a post or sentence, as political ideologies tend to be
more fluid, rhetorical, and generally difficult to classify even by humans. An extension of the work above would be to train models for political ideology (e.g., liberal
versus conservative), rather than positive or negative sentiment. Such work has
already been attempted in analyzing political blogs (Durant and Smith 2006), but
merits further rigorous research. An approach similar to the one taken in this
paper would be ideal.
67
6. Conclusion
This chapter targets an entity-centric document-level sentiment classification of
political blogs. We presented the results of an initial study that sought to investigate the feasibility of combining linguistic indicators of political sentiment with
non-linguistic information obtained from social network analysis. Using crowdsourced sentiment annotations centered around Barack Obama during the 2008
U.S. presidential election sampled from a large corpus of 2.8 million blog posts,
a hybrid machine-learning and logic-based framework was employed which
relies on standard shallow document classification, deep linguistic multi-entity
sentiment analysis and scoring, and social-network modeling. The initial results
demonstrate the complexity of the task, and point towards the positive effects of
learning features that exploit entity-level sentiment scores and social network
structure.
Acknowledgements
We would like to thank the Predictive Modeling Group at IBM for providing us with the blog
data, and Nigel Crook at Oxford University Computing Laboratory for supporting code.
References
Adamic, L. and N. Glance. 2005. The political blogosphere and the 2004 US election: divided
they blog. In Proceedings of the 3rd International Workshop on Link Discovery (LinkKDD
2005, New York, NY), pp. 3643.
Clauset, A., M.E.J. Newman and C. Moore. 2004. Finding community structure in very large
networks. Physical Review E, 70(6), 06611, 6 pages.
Csardi, G. and T. Nepusz. 2006. The igraph software package for complex network research.
International Journal Complex Systems, 1695.
Durant, K.T. and M.D. Smith. 2006. Mining sentiment classification from political web logs. In
Pro-ceedings of Workshop on Web Mining and Web Usage Analysis of the 12th ACM SIGKDD
International Conference on Knowledge Discovery and Data Mining (WebKDD-2006).
Durant, K.T. and M.D. Smith. 2007. Predicting the political sentiment of web log posts using
supervised machine learning techniques coupled with feature selection. In Advances in
Web Mining and Web Usage Analysis: Proceedings of the 8th International Workshop on
Knowledge Discovery on the Web (WEBKDD 2006, Philadelphia, PA), pp. 187206.
Efron, M. 2004. Cultural orientation: Classifying subjective documents by co-citation analysis.
In Style and Meaning in Language, Art, Music, and Design: Papers from the 2004 AAAI Fall
Symposium, Arlington, VA. Technical Report FS-04-07, pp. 4148.
Hall, M., E. Frank, G. Holmes, B. Pfahringer, P. Reutemann and I.H. Witten. 2009. The weka
data mining software: An update. ACM SIGKDD Explorations Newsletter 11(1), pp. 1018.
DOI: 10.1145/1656274.1656278
Hargittai, E., J. Gallo and M. Kane. 2008. Cross-ideological discussions among conservative and
liberal bloggers. Public Choice 134(1), pp. 6786.
Hsueh, P.Y., P. Melville and V. Sindhwani. 2009. Data quality from crowdsourcing: a study of
annotation selection criteria. In Proceedings of the NAACL HLT 2009 Workshop on Active
Learning for Natural Language Processing, pp. 2735. DOI: 10.3115/1564131.1564137
Kim, S.-M. and E. Hovy. 2007. Crystal: Analyzing predictive opinions on the web. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and
Computational Natural Language Learning (EMNLP-CoNLL, Prague), pp. 10561064.
Kittur, A., E.H. Chi and B. Suh. 2008. Crowdsourcing user studies with Mechanical Turk. In
Proceeding of the 26th annual SIGCHI conference on Human factors in computing systems,
Florence, pp. 453456.
Kwon, N., S.W. Shulman and E. Hovy. 2006. Multidimensional text analysis for erule-making.
In Proceedings of the 7th Annual International Conference on Digital Government Research
(DG.O 2006, San Diego, CA), pp. 157166.
Lerman, K., A. Gilder, M. Dredze and F. Pereira. 2008. Reading the markets: Forecasting public
opinion of political candidates by news analysis. In Proceedings of the 22nd International
Conference on Computational Linguistics (COLING 2008, Manchester), pp. 473480.
Leskovec, J., M. McGlohon, C. Faloutsos, N. Glance and M. Hurst. 2007. Cascading behavior
in large blog graphs: Patterns and a model. Society of Industrial and Applied Mathematics:
Data Mining (SDM07, Minneanapolis, MN). Tech report (12 pp.): CMU-ML-06-113.
Lin, W.H., T. Wilson, J. Wiebe and A. Hauptmann. 2006. Which side are you on? identifying
perspectives at the document and sentence levels. In Proceedings of the 10th Conference on
Computational Natural Language Learning (CoNLL-X New York, NY), pp. 109116.
Malouf, R. and T. Mullen. 2008. Taking sides: User classification for informal online political
discourse. Internet Research 18(2), pp. 177190. DOI: 10.1108/10662240810862239
Manning, C.D., P. Raghavan and H. Schtze. 2008. An Introduction to Information Retrieval.
Cambridge: CUP. DOI: 10.1017/CBO9780511809071
McPherson, M., L. Smith-Lovin and J.M. Cook. 2001. Birds of a feather: Homophily in social
networks. Annual Review of Sociology, 27(1), pp. 415444. DOI: 10.1146/annurev.
soc.27.1.415
Moilanen, K. and S. Pulman. Multi-entity sentiment scoring. 2009 In Proceedings of the Recent
Advances in Natural Language Processing (RANLP 2009, Borovets, Bulgaria), pp. 258263.
Mullen, T. and R. Malouf. 2006. Preliminary investigation into sentiment analysis of informal
political discourse. In Computational Approaches to Analyzing Weblogs: Papers from 2006
AAAI Spring Symposium (Stanford, CA), pp. 159162.
Pang, B. and L. Lee. 2008. Opinion Mining and Sentiment Analysis, volume 2 of Foundations and
Trends in Information Retrieval. Now Publishers.
Porter, M. 1980 An algorithm for suffix stripping. Program 14(1), p 3.
Snow, R., B. OConnor, D. Jurafsky and A.Y. Ng. 2008. Cheap and fastbut is it good?: evaluating non-expert annotations for natural language tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2008, Honolulu, HI),
pp. 254263.
69
Thomas, M., B. Pang and L. Lee. 2006. Get out the vote: Determining support or opposition
from congressional floor-debate transcripts. In Proceedings of the Conference on Empirical
Methods in Natural Language Processing (EMNLP 2006, Sydney, Australia), pp. 327335.
Tremayne, M., N. Zheng, J.K. Lee and J. Jeong. 2006. Issue publics on the web: Applying network theory to the war blogosphere. Journal of Computer-Mediated Communication 12(1),
pp.290 310. DOI: 10.1111/j.1083-6101.2006.00326.x
Van Atteveldt, W., J. Kleinnijenhuis and N. Ruigrok. 2008a. Parsing, semantic networks, and
political authority using syntactic analysis to extract semantic relations from Dutch newspaper articles. Political Science 16(4), pp. 428446.
Van Atteveldt, W., J. Kleinnijenhuis, N. Ruigrok and S. Schlobach. 2008b. Good news or bad
news? Conducting sentiment analysis on Dutch text to distinguish between positive and
negative relations. Journal of Information Technology & Politics 5(1), pp. 7394. DOI:
10.1080/19331680802154145
Yu, B., S. Kaufmann and D. Diermeier. 2008. Exploring the characteristics of opinion expressions for political opinion classification. In Proceedings of the 9th Annual International
Conference on Digital Government Research, Partnerships for Public Innovation, Vol. 289
(DG.O 2008, Montreal), pp. 8289.
chapter 4
Issue framing and language use

intheSwedish blogosphere
Changing notions of the outsider concept
Department of Political Science, University of Gothenburg

and Gavagai, Stockholm
Issue framing has become one of the most important means of elite influence on public opinion. In this paper, we introduce a method for investigating
issue framing based on statistic analysis of large samples of language use. Our
method uses a technique called Random Indexing (RI), which enables us to
extract semantic and associative relations to any target concept of interest, based
on co-occurrence statistics collected from large samples of relevant language
use. As a first test and evaluation of our proposed method, we apply RI to a
large collection of Swedish blog data and extract semantic relations relating to
our target concept outsiders. This concept is widely used in the public debate
both in relation to labour market issues and socially related issues.
Introduction1
Issue framing has become one of the most important means of elite influence on
public opinion. According to prior work, we understand issue framing as a process
where a communicator defines and constructs a political issue or public controversy by emphasizing a subset of potentially relevant considerations and [thereby
pointing to the] essence of the issue (Slothuus 2008; Slothuus and de Vreese 2008).
However, we still have a limited understanding of how arguments and rhetoric
from political parties actually influence the formation of political opinion. We
know that elites attempt to influence opinions by framing issues, i.e., by presenting alternative descriptions and interpretations of different issues. A great body
1. The authors are in great debt to Rebecka sbrink and Henrik Lindholm at the Department
of Political Science, University of Gothenburg, for their assistance with collecting and analyzing
the party related documents that form the basis for the qualitative part of this study.
72
of literature has also demonstrated the impact of issue frames on public opinion
(Chong and Druckman 2007; Kinder 2003).
We believe that an important key towards a deeper understanding of framing
theory is to view framing as a dynamic process in which issues are continuously
refocused and redefined by language use. Parties and commentators are actively
framing issues by the very words they choose to speak about it; speaking about
X in terms of Y can send a completely different message than speaking about
it in terms of Z. It is well known that language use may influence the outcome of
policy debates; the chances of obtaining ones goals can be dramatically improved
by getting everyone to debate an issue in your terms (Naurin et al. 2009). The
goal of any skilled advocate is to get her idea to catch on, to reach the tipping point
where her way of thinking is not just one way of thinking, but the way of thinking.
In this chapter, we introduce a method for investigating issue framing based
on statistical analysis of large samples of language use. Our method uses a technique called Random Indexing (RI), which enables us to extract semantic and
associative relations to any target concept of interest, based on co-occurrence statistics collected from very large samples of relevant language use. If X and Y
both occur together with Z, our method will relate X and Y. We suggest that
such semantic relations are indicative if not constitutive of framing, and that
this type of quantitative analysis therefore is an attractive method for investigating
issue framing.
As a first test of our proposed method, we have applied RI to a large collection of Swedish blog data from the period 2008 to 2010, and extracted semantic
relations to a target concept referring to the notion outsiders (utanfrskap in
Swedish), which is a concept that has been widely used in the public debate by
parties, commentators, etc., especially since the national election of 2006. In the
public debate, the concept is often used in relation to labour-market issues but also
to socially related issues. This paper exemplifies and discusses the various types of
relations, and their implications within issue-framing theory. The analysis starts
with a qualitative/quantitative approach where we have been tracking the language
use for the concept of outsiders among the two dominant and opposing parties
in Swedish politics, the Social Democratic Party and the Conservative Moderate
Party. The results from this part of the study are then used as a bench-mark for a
deepening understanding of the framing of the issue in the blogosphere.
Chapter 4. Issue framing and language use intheSwedish blogosphere
The case of Sweden: Issue framing and the outsider concept

For several decades the Social Democratic Party has been dominant in Swedish
politics. With only a few interruptions, in 1976 and in 1991, the Social Democratic
Party has been governing with support from the former Communist Left Party
and the Green Environmentalist Party. However, during the election of 2006 the
chain was broken when the Conservative Moderate Party together with the Center
Party, the Liberal Party and the Christian Democratic Party formed an alliance of
bourgeoisie parties. With a common platform, the bourgeoisie parties succeeded
in breaking the Social Democratic dominance. For the first time in Swedish
modern history, they also managed to be reelected for government in 2010. One,
amongst many, explanations for the success of the bourgeoisie parties is that they
managed to take over the ownership of the employment issue, which is an issue
that traditionally has been intimately connected with the Social Democratic Party
(Oscarsson and Holmberg 2008).
The literature offers few theoretical explanations for how such a sudden shift
in issue ownership could occur (see Walgrave and De Swert 2007 for a review
of the subject). In this respect, we believe that policies and issues should not be
thought of as being fixed but rather, they are dynamic in the sense that an issue is
defined by how parties and commentators speak about it. The language use in the
public debate might thus affect voters perceptions of parties issue ownership; e.g.,
which party is perceived as being most competent to handle the issues at stake.
In order to evaluate the applicability and functionality of the RI-model for the
purpose of text analysis in general and framing theory in particular, we have tried
to identify policy issues and domains where we can expect different language use
or frames to be present. We will focus on a much debated issue in Swedish politics
during recent years known as outsiders.
The term outsiders has been present in Swedish politics for some ten years, in
which period it was used in relation to different forms of outsiderness/alienation.2
However, during the Swedish national election in 2006, the term was reintroduced
as a targeted concept by the alliance of bourgeoisie parties, who turned it into a
prioritized goal to decrease the number of outsiders in Swedish society. Their
2. The term outsiders may thus, in general terms, refer to situations where individuals or
groups of individuals are or are experiencing that they are excluded from a group or community
that is perceived as being desirable to be a part of. The concept may thus pertain to groups of
varying sizes that are not accepted in a society, such as different forms of minority groups that
are not reckoned or accepted as belonging to the normal society, which in turn will undermine
their opportunities to participate to the same extent as the majority population.
73
74
definition of the concept was now referring to broad unemployment.3 This link
between outsiders and broad unemployment has in turn been heavily criticized,
especially among opposition parties.4 The critique has mainly been about matters
such as which segments of the population should be classified into the category of
broad unemployment and whom should not.
In the academic literature, the outsider concept often refers to unemployment
and pertains to groups of citizens outside the labor market (see fc. Rueda 2005;
Rueda 2006) and this is how the bourgeoisie parties have been referring to the
concept. Nevertheless, the term outsider remains a nuanced and imprecise term
and the labor market is only one arena for outsiderness. One can in this respect
also speak about social, cultural or political outsiders, as examples of outsiderness.
This has also been the case in Swedish politics, where it traditionally also has been
used in relation to issues of segregation and integration.
The fact that the concept has been used in various contexts at the same time,
and the fact that the bourgeoisie parties never gave any clear definition when they
reintroduced the term outsider, makes the notion imprecise and multiplex. This
makes the term, or rather the usage of the term outsider in the Swedish public
debate, a fruitful notion for analyzing framing effects. The hypothesis is that the
outsider concept is used and spoken about in different terms by different actors
and in different contexts. How exactly the concept is used and understood in the
public debate is, of course, an empirical question.
Methodological considerations
Our analysis of how the outsider concept is framed consists of both a qualitative/quantitative and a purely quantitative part. The purpose of the qualitative/
quantitative analysis is to create a background and a benchmark for how the two
main political alternatives in Swedish politics frame outsiders. For this part of
the analysis we gathered official documents, articles, news-articles and speeches
from the parties official web-pages. For the Social Democratic Party, a total of 195
documents in which the term outsiders is explicitly mentioned was found during
3. Broad unemployment is in turn defined as the part of the population between 1664 years
of age minus the amount of gainfully employed and minus the amount of students that were not
seeking employment, divided by the entire population within 1664 years of age (source: http://
www.slideshare.net/guest3fc6c8/utanforskap-2009-moderaternas-egen-definition).
4. See fc. http://www.svd.se/opinion/brannpunkt/thomas-ostros-utanforskapet-vaxer_
4459805.svd
the time-span from 2008 and onwards. For the Conservative Moderate Party, the
corresponding number was 148 documents from 2001 to present day.
The qualitative analysis was done in two steps. First, we aimed for an unbiased
skim through the documents in order to obtain knowledge about the contexts in
which the word outsider appears. From the public debate we knew that we could
expect the outsider concept to be used in relation to the labor market; however,
in common lexical definitions it can also be related to culture, language and geographical segregation. We have tried to be as explicit as possible by including
snippets directly from the texts in order to illustrate our findings and by listing
the original Swedish snippets in the Appendix. The second step was to quantify
the revealed relationships in order to get a more systematic understanding of the
contexts in which the outsider concept appears most frequently.
The second, quantitative, part of the study applies RI to a large data set of
Swedish blogs in order to extract semantic relations to the outsider concept.
These semantic relations provide information about the words that are most frequently related with the outsider concept in the blogosphere i.e., how the issue
has been framed in this particular data set. This gives us an indication of how the
concept is framed in the current debate, and how the frames are shifting in the
blogosphere. However, such analysis will not by itself be able to automatically
tell or determine to what extent these frames are salient in the blogosphere nor
whether these frames are directly related to the frames used by any of the political
parties. In order to establish this, we need to manually compare the RI similarities
to the benchmark analysis.
Random Indexing
Random Indexing (RI; Kanerva et al. 2000; Sahlgren 2005) is a statistical textanalysis technique that can be applied to massive amounts of text data in order
to extract semantic and associative relations between words. The RI technique is
a specific implementation of a word-space model (Schtze 1993), which is a breed
of computational semantic models that use co-occurrence statistics to compute
similarity between words. These models represent words by high-dimensional
context vectors, such that each dimension represents a particular context, and
each element indicates the (normalized) frequency of occurrence of the word
in that particular context. This means that words that have occurred in similar
contexts get similar context vectors, and that the context vectors therefore can
be used to compute similarity between words using standard vector similarity
metrics. These similarities are interpreted as indicating semantic or associative
relations, depending on what type of context is used in collecting the occurrence
75
76
information; if word types are used as context, the model will extract semantic
similarities, while if documents are used as context, it will extract associative
similarities (Sahlgren 2006).
As an example of how word spaces can be used to investigate language use,
imagine we are interested in how the word labour has been used in a large collection of texts on political issues. Applying a word-space model on this data can
tell us both which words are used in similar ways (e.g. employment and job) and
which words have an associative relation with labour in this particular data (e.g.
welfare and security). Together these analyses give us a good understanding of
the usage of words, and it allows us to find text-specific similarities and associations that will not be present in standard lexical or conceptual resources. This is
particularly useful when dealing with very productive and dynamic text styles like
those typically encountered in social media and the blogosphere.
Word-space modeling has become a standard method in natural language
processing for capturing word usage, and models have proven their mettle in an
impressive range of large-scale linguistic learning tasks and text analysis applications, including automatic thesaurus construction, terminology mining, word
categorization, word sense disambiguation, document clustering, knowledge assessment, text categorization, information retrieval, and modeling of various behavioral
effects in psycholinguistic and cognitive experiments (see, e.g., Turney and Pantel
[2010] for an overview of NLP methods and applications). However, word-space
models tend to suffer from scalability and efficiency issues due to the heavy algebraic machinery involved (e.g., issues regarding very high dimensionality, or the
use of matrix decomposition techniques). RI, on the other hand, was developed
specifically to overcome problems with scalability and efficiency when dealing with
high-dimensional data. Instead of representing each context as a separate dimension, which inevitably leads to very high-dimensional models that are susceptible
to both scalability and efficiency issues, RI uses fixed-dimensional vectors in which
each context is represented by a small number of randomly chosen dimensions.
Every time a word occurs in a context, all the elements representing that context is
incremented in the words fixed-dimensional context vector. This use of distributed
representations obliterates the need for dimension reduction, and ensures that the
dimensionality of the context vectors never increases, even if the data continuously
does. We refer to Sahlgren (2005) for a more thorough introduction to RI.
As mentioned above, our evaluation of the applicability of RI for investigating
framing effects is operationalized in several steps. The first step consists of a qualitative study of how the term outsiders is used and spoken about on the official
party websites of the two largest and opposing parties: the Social Democrats and
the Conservative Moderate Party. The second step applies RI to a 1.5 billion-word
database of Swedish blog texts, collected between November 2008 and September
2010, and provided by the Swedish blog search engine Twingly (www.twingly.
com). The purpose of this step is to investigate which words are mostly associated
with outsider in the public blogosphere. We also try to identify which of the parties language use is most dominant in the blogosphere.
The fact that we only use data from blogs and from the parties web-pages
implies that the results should not be considered to be valid for the general public
debate as such. The main purpose with this specific study is not to maximize the
external validity, but rather to evaluate the prospects for using RI for investigating
framing effects empirically on large amounts of data. By focusing on the blogosphere we have the opportunity to do this with data that are easily accessible.
Language use by the Social Democratic and the Conservative Moderate

Party in relation to outsiders5
The Conservative Moderate Party
The Conservative Moderate Party is the largest of the bourgeoisie parties in
Swedish politics and since the election of 2006 they have formed the government
in an alliance with the three remaining bourgeoisie parties (the Center Party, the
Liberal Party and the Christian Democratic Party).
The results from our qualitative analysis indicate that for the Conservative
Moderate Party, the word outsiders seems to be synonymous with unemployment.
In documents, newsletters and speeches published on their webpages, the word
outsiders generated a total of 148 hits during a time span between fall of 2008 to
September 2010. The word appears together with unemployment (Examples 1
and 2), early-retirement pension (Example 3), the opposition, health insurance
and disability pension:

(1) The goal with the change in the system is to help people to get back into
employment and decrease the gap between outsiders and employment.6
(2) We choose to carry out understandable policies for full employment and fewer
outsiders.7
5. The quotes in this section are translated into English to facilitate readability. The original
Swedish text snippets are listed in the Appendix.
6. Article published 2010-03-01. http://www.moderat.se/web/Kortare_vagar_tillbaka.aspx
7. Published2009-09-15.http://www.moderat.se/web/Fokus_pa_jobb_och_valfard_i_regerings
forklaringen_1.aspx
77
78
(3) During the Social Democrats time in power there was a dramatic increase
in sick leave. To improve the stats they transferred 140 persons each day into
early retirement pension and thereby labeled them as permanent outsiders.8
Of the publications that mention outsiders a vast majority focuses on a group that
is the opposite of employed, while a few publications aim to fuse it with the opposition. A distinction in this respect is that the news and newsletters lean more towards
the unemployment aspect while the speeches are more focused on the opposition.
Only a hand full of the documents deviate from the use of the term outsiders in relation to unemployment. In some examples the term outsiders is spoken
about in national contexts, as opposed to an EU membership context, outsider
as the opposite of membership of the EMU. In a few instances where the term
outsider is used in relation to other issues, such as schools, fighting crime, senior
citizen retired out of old age and issues concerning segregation and integration, it
is still used as an indicator of those who are not employed. On the topic of criminality among under aged, Example (4) is taken from an article by Jan R Andersson
and Krister Hammarbergh, both members of the Conservative Moderate Party
and of the Swedish Parliament, published in Vsterviks-Tidningen, Vimmeby
Tidning, Oskarhamns-Tidningen and Kinda Posten.

(4) It is necessary to see what causes criminality: often it is unemployment, lack

of social fellowship and lack of security. This is why we need to work even
harder to decrease the number of outsiders: higher employment, new jobs,
better school and to integrate the immigrants and refugees. Through these
actions, we can provide for more children to have a secure childhood the
most efficient preventive action against criminality. By working for a society
with fewer outsiders we also work against crime. This is a connection so
strong, that it sometimes is forgotten.9
Another Example (5) of how outsiders are used in relation to crime comes from
the questions and answers section of the webpage.

(5) Crime is fought by a strong justice system and policies for fewer outsiders and
less unemployment.10
Cristina Husmark Pehrsson, Minister of Welfare and the minister responsible for
Nordic cooperation, uses the term outsiders when discussing the economic terms
for senior citizens (Example 6).
8. Article published 2009-10-26. http://www.moderat.se/web/Fran_sjukskriven_till_arbete_1.
aspx
9. http://www.moderat.se/web/Nyheter_1565.aspx
10. http://www.moderat.se/web/2a8a2a45-c5ba-466e-b42d-0d0fcac07267_2_1_1.aspx
(6) A high number of outsiders threatens the welfare since fewer contribute to
create the limited resources that have to be shared by more people. The governments efforts for high employment are the only long term sustainable way
to protect the welfare. (---) The measures that have been taken to lower the
number of outsiders also contribute to secure the pensions.11
As these quotes suggest, even though being used in various contexts, the term
outsiders mainly refers to people who for various reasons are outside the labor
market. In several cases this connection is implicit; get outsiders into employment
and they will cease being outsiders. Concluding, the Conservative Moderate Party
in general is fairly unambiguous in their use of the term outsiders. Mainly it is
referred to in terms of employment.
The Social Democratic Party

During the time-span between 2008 until today, the Social Democratic Party
is mainly using the word outsiders in a similar manner as the Conservative
Moderate Party does. However, an interesting finding, when stretching focus
some more years back in time, is that the party earlier tended to use the concept
in a row of shifting contexts. For example, before the parliamentary election
2002 the Social Democrats mentioned the concept of outsiders in their election pamphlet about integration and diversity.12 Mainly, the party considered
the word outsiders connected to immigration, segregation and discrimination
during this period.
Four years later, the party still talked about outsiders in connection to immigration, segregation and discrimination.13 But in 2006, the Social Democrats also
talked about outsiders in connection with crime and safety.14
In 2007 the party starts to turn the phrase. Like the Conservative Moderate
Party, the Social Democrats have begun to mention outsiders in connection with
employment and joblessness. But, only when referring to the definition of outsiders as a creation of the Conservatives and with a view to criticize the right-wing
government. In her opening speech introducing the party leader debate of the
11. http://www.moderat.se/web/Fler_i_arbete_raddar_pensioner.aspx
12. http://www.socialdemokraterna.se/upload/val/val_02/integration.pdf
13. http://www.socialdemokraterna.se/Vart-parti/Socialdemokratiska-riksdagsgruppen/LucianoAstudillo/nyheter/Arkiv/Artiklar/Artiklar-arkiv/Artiklar-riksdagsaret-200607/Integrationspolitikutan-tanke-/
14. http://www.socialdemokraterna.se/upload/val/val_06/valblad06/valblad_060828/valblad%205_
trygghet%20mot%20 brott_060821.pdf
79
80 Stefan Dahlberg and Magnus Sahlgren
13th of June 2007, Mona Sahlin, party leader of the Social Democrats criticized
the government for cutting the unemployment benefit fund.15
In the summer of 2008, the concept of outsiders became more and more
associated with jobs, welfare, and safety. Here is a speech by Mona Sahlin in
Vitabergsparken, Stockholm:

(7) What is the answer from the government Reinfeldt today? In June the outsiders
increased for the third month (---) Its been half a term of office. Its time for
the evaluation. It is time for proofs. Have they done what they intended to?
What about the jobs? What about the outsiders? What about the welfare and
safety?16
One year later, during the summer of 2009, the adoption of the definition is
completed. The Social Democrats now speak of outsiders and employment or
unemployment. Often without referring to the definition as a creation of the conservatives (Example 8, Mona Sahlin during the week of politics in Almedalen,
Sweden):17

(8) The employment rate must increase among the youth, among immigrated
Swedes, among part-time working women, and, never the least, amongst the
elderly. To reach that goal the absence of freedom called joblessness and outsiderness must be defeated and for that matter, the involuntary part-time
unemployment. If we want to get there, the concept of the working line must
apply to everybody.
On the 29th of October 2009, Mona Sahlin wrote in an article on the official homepage (Example 9).18

(9) During the last year, the numbers of unemployed have increased with 100000.
Since the election in 2006 the outsiders increased with 70000 (---) For the
election of 2010, the joblessness will be the most important issue.
15. http://www.socialdemokraterna.se/upload/MonaSahlin/Dokument/Mona%20Sahlins%20
anf%c3%b6rande%20i%20riksdagens%20partiledardebatt.pdf
16. http://www.socialdemokraterna.se/Mona-Sahlin/Tal/2008/Mona-Sahlins-sommartal-iVitabergsparken-2008/
17. http://www.socialdemokraterna.se/Mona-Sahlin/Tal/2009/Mona-Sahlin-anforandeAlmedalen-2009/
18. http://www.socialdemokraterna.se/media/pressarkivet/nyhetsarkivet-2001--/mona-sahlinjobben-ar-viktigast/
However, when the Social Democrats want to talk about perceived consequences of
the politics formed by the right-wing alliance, they focussed on the social angle of
outsiderness. Example (10) is a cut from an article by eight politicians, six from the
Social Democratic Party, one from the Greens and one from the Left Party. The article was published in the Swedish newspaper Corren on the 26th of February 2010.19
(10) Four years ago, the right-wing parties won the election on their promise of
more employment and decreasing outsiderness. Unfortunately, when we now
evaluate the term of office, the result is depressing. The government throw
out tens of thousands Swedish citizens from health insurance to a reality
with no insurance at all. In just six months the numbers of unemployed have
increased with more than 100000 persons, and numbers of outsiders with
70000. Receivers of public assistance are calculated to increase with 50 percent
between 2006 and 2011.
And Example(11) is another snippet of news from the Social Democratic homepage, published on the 4th of March 2010.20
(11) Today, statistics from Statistics Sweden showed that the amount of outsiders,
the Conservatives own measure for how well the politics work, have increased
with about 70 000 between 2006 and 2009. Yesterday, statistic showed that
public assistance, the most serious kind of outsiderness, have increased in
more than 90 percent of the municipalities last year.
Historically, the language use among the Social Democratic Party has not been as
clear cut as in the case of the Conservative Moderate Party. Instead it seems that
for the Social Democrats, the word outsiders has been used in shifting contexts
that gradually changed over time. Traditionally, they often tend to speak about
outsiders both in terms of unemployment, as well as, to a rather large extent, in
relation to social outsiderness and integration of citizens with their roots in foreign countries. However, from 2008 onwards, the Social Democrats tend to use the
term in a similar way as the Conservative Moderate Party, i.e. mainly in relation
to unemployment and labor-market issues. This is an interesting finding by itself
to the extent that the Social Democrats, over time, have started to speak about
outsiders by the same terms as the Conservative Moderate Party. From a framing
perspective, the Conservative Moderate Party has thus managed to make their
language use on this specific issue to become the dominant frame for outsiders.
19. http://www.socialdemokraterna.se/Vart-parti/Socialdemokratiska-riksdagsgruppen/SoniaKarlsson/nyheter/Skapa-ett-nytt-trygghetssystem/
20. http://www.socialdemokraterna.se/media/nyheter/70-000-fler-i-utanforskap-med-moderaterna/
81
82
In an attempt to more systematically compare language use by the Social

Democratic Party and the Conservative Moderate Party in relation to the outsider
concept, we have also performed a simple quantitative content analysis based on
the documents from the parties web pages.
From quality to quantity in party related documents

The national elections for the Swedish Parliament were held on September 19th,
2010. Table 1 shows the number of party documents containing the word outsiders and the context of outsiders between 2008 and 2010.
Table 1. Amount of party documents containing the word outsiders and the context
of outsiders between 2008 to 2010 (percentages).
outsider+unemployment
outsider+employment
outsider+work
outsider+jobs
outsider+subsidy
outsider+social
outsider+welfare
outsider+safety
outsider+insurance
outsider+segregation
outsider+integration
outsider+housing
outsider+povery
outsider+criminality
outsider+unsafety
outsider+abuse
outsider+insecurity
N: (contexts)
N: (documents)
Index of dispersion
Social Democratic Party
Conservative Moderate Party
2008 2009 2010 Total
2008 2009 2010 Total
4
9
17
23
9
1
14
9
3
1
1
3
4
1
3
0
0
4
9
19
22
6
3
12
10
4
1
2
2
4
0
0
1
0
6
7
17
21
6
4
12
9
2
3
2
3
3
1
3
1
0
5
8
17
21
6
3
12
9
2
2
2
3
4
1
2
1
0
102
34
.93
134
39
.92
347
122
.94
583
195
.94
0
8
23
18
7
10
7
8
0
0
5
0
5
5
3
2
0
1
6
27
15
8
1
10
8
2
4
2
2
1
7
2
3
1
2
9
26
26
6
2
8
10
0
1
3
2
1
1
1
1
1
1
8
26
20
7
3
8
9
1
2
3
2
1
4
2
2
1
61
19
.93
137
61
.92
168
68
.89
366
148
.91
Comment: The analysis is made as a simple word count for each year based on electronic documents
available by the parties web pages (www.socialdemokraterna.se and www.moderat.se). The percentages
are based on the number of contexts. The contexts are in turn collected from the results of the qualitative
content analysis of party documents. Index of dispersion is a simple measure for dispersion where a value
of 1 indicates a totally even distribution of observations in each category while a value of 0 indicates the
2
opposite. The measure is calculated as: 1 p , where p is proportion while k stands for category.
(k 1) / k
Apparently, the number of documents containing the word outsiders increased

during 2010 for both parties, probably as a result of a growing number of documents in general during the election year. Clearly there is no sharp division in
language use between the two parties. The word outsider is mainly used in relation to employment issues and in relation to welfare and social security. In general,
the Social Democratic Party is using the term outsiders more frequently than are
the Conservative Moderate Party. The language use in relation to the outsider
concept is, however, particularly stringent for both parties. When comparing
the dispersion measures for the distribution in each category, the Conservative
Moderate Party is a tiny bit less ambiguous in their language use during 2010 but
the differences are still very small. To some sense, it seems as that the main difference is that the Social Democratic Party to some greater extent tend to speak about
outsiders in relation to the concepts of insurance and poverty.
Random Indexing of words related to outsider in the Swedish

blogosphere 20082010
In order to get an idea of how the word outsider has been used in the Swedish
blogosphere in the period from 2008 to 2010, we used RI to produce 1000-dimensional word spaces quarterly between November 2008 and September 2010.21 For
these experiments, we used word types as contexts in order to extract semantic
relations from the word space, since we are interested in the meaning of the target
term. Occurrence information was collected within a context window that spans
two proceeding and two succeeding words, and ignores words with frequency less
than 10. Table 2 shows the ten most semantically similar terms and their cosine
similarities to outsider for each quarter.
During the first four quarters from 2008 to late 2009 the outsider concept
seems to some extent to be related to housing situations by terms such as overcrowding, waiting lists or lack of housing. This is an interesting finding indeed
since it, to a large extent reflects the way the Social Democratic Party was talking
about outsiders during earlier years between 2002 until 2008, that is in terms of
segregation in the suburbs. In this respect, the results from our qualitative content
analysis revealed an interesting pattern in that the language used by the Social
Democrats during the last eight years subsequently adapted to the language used
21. The decision to conduct the studies on a quarterly year basis is more or less an arbitrary
decision. With shorter time periods one risks to receive too many reference points with too little
variation in, while longer time periods may contribute with variation but being less precise in
discovering more sudden shifts in language use in the blogosphere.
83
84
Table 2. Words related to the term outsider in the Swedish blogosphere between 2008 to 2010.
2008.Q4
Outsider:
Prox.
2009.Q1
Outsider:
Prox.
2009.Q2
Outsider:
Prox.
2009.Q3
Outsider:
Prox.
fertile ground:
insecurity :
exclusion:
surplus:
safety net:
overcrowding:
network:
conditions:
fear:
criticized:
.25
.24
.22
.20
.19
.18
.18
.18
.18
.18
long-term unemployed:
inscribed:
induced:
overcrowding:
taken cared of:
waiting lists:
contacted:
poverty :
crime:
predictions:
.34
.34
.22
.21
.20
.15
.15
.15
.14
.14
subsidy dependency :
contribution line:
segregation:
poverty :
unemployment:
prosperity :
lack of housing:
labor shortage:
homelessness:
insecurity :
.39
.33
.30
.26
.25
.24
.23
.23
.23
.22
poverty :
job-quake:
admin. burdens:
structural:
number of students:
range:
lack of housing:
segregation:
insecurity :
income inequalities:
.33
.31
.28
.26
.25
.23
.23
.23
.21
.20
2009.Q4
Outsider:
Prox.
2010.Q1
Outsider:
Prox.
2010.Q2
Outsider:
Prox.
2010.Q3
Outsider:
Prox.
job-quake:
unemployment:
health insurance:
fewer:
poverty :
vulnerability :
security :
alliance:
addiction:
.35
.30
.28
.25
.25
.22
.22
.22
.21
.21
long-term unemployed:
alliance:
unemployment:
overlooked:
welfare:
segregation:
moderate-led:
fewer:
job-quake:
government:
.25
.25
.22
.21
.21
.20
.20
.20
.20
.19
need of sleep:
consumption space:
welfare dependency :
risk-taking:
poverty :
segregation:
needs:
influence:
energy intake:
.33
.31
.28
.26
.26
.26
.26
.25
.24
.24
poverty :
need of sleep:
adjustment insurance:
employment supply :
insecurity :
crime:
citizen participation:
significantly :
community policing:
.43
.30
.24
.24
.24
.23
.23
.22
.22
.22
Comment. The table shows the top-ten neighboring words to the concept of outsiders. The division of the time-periods is done quarterly from October 2008 to September
2010. The proximity measures are the cosine angles between the context vectors. It should be noted that every time an experiment is run using RI, the resulting word space will
be slightly different, partly due to the impact of parameter-settings and partly due to the very fact that we are using random indexing as a base for the experiment. This means
that: (1) that the similarity measures are highly relative (and, hence, only informative in relation to other similarity measures in the same model) and (2) that there may be some
variance in the analysis. One way to overcome these issues is to run the simulations multiple times with different random seeds and different parameters, and only report the
robust results. The original Swedish terms are listed in the Appendix.
by the Conservative Moderate Party when speaking about outsiders. The result
from the RI analysis of the earlier time-periods of the blogosphere could thus be
an indication that the language use in the blogosphere is to some extent lagging
behind in relation to the language used by the parties in general and for the Social
Democratic Party in particular, given that the latter are the agenda setters in this
respect. The concepts relating to housing situations is, however, not used to any
greater extent in relation to the outsider concept by any of the parties during the
investigated period for the RI analysis in any of neither documents.
Another indication of the impact of the Social Democratic language use during 2008 and early 2009 is the occurrence of the term poverty, which is one of
the most closely related terms to the outsider concept (Table 2). In the previous
analysis of the party documents there were some indications that the concepts of
poverty and insecurity to some extent were more frequently used in relation to
outsiders by the Social Democrats. The presence of terms relating outsiders to
concepts of housing situations, poverty and segregation, etc., should, however, not
be exaggerated since the most widely related concepts by far pertain to subsidy
dependency and long-term unemployment, which are concepts that are mainly
used by both parties in relation to outsiders over this period.
However, especially from late 2009 onwards, words such as job-quake, contribution line, subsidy dependency and Alliance score higher in the word space.
Interestingly, these terms are directly related to concepts invented and used by the
Conservative Moderate Party and its alliance parties. At the same time, words such
as poverty and segregation are scoring lower. This could be an indication that
the language use by the Conservative Moderate Party and its alliances are becoming the agenda-setting language when it comes to defining the outsider concept
before the election campaign.
Nevertheless, based on the results from the quantitative content analysis of the
party documents, we cannot discern any sharp divisions in the language use in
relation to the outsider concept by any of the two parties during the investigated
time period from late 2008 onwards. There are some indications that the Social
Democrats to some extent are a bit more ambiguous in their language use. For
example, they are, as mentioned, using the outsider concept in relation to poverty
and insecurity. However, in general both parties tend to speak about outsiders
in relation to the labor market. A striking difference is, however, that while the
Social Democratic Party tends to speak about employment and unemployment
the Conservative Moderate Party tends to speak more one-sidedly of employment.
Both these terms are also apparent in the RI word space. Nevertheless, given the
results from prior analysis of the party related documents it is difficult, not to say
impossible, to validate which parties language use is dominating the blogosphere
during the investigated time period.
85

Table 3. Proximity measures for the concept of outsiders and the Conservative Moderate Party and the Social Democratic Party
in the blogosphere between 2008 and 2010.
2008.Q4
Freq.
Prox.
moderate + outsider
social democrat + outsider
122 (/3961)
74 (/4141)
0.11
.08 (.11)
149 (/7134)
184 (/8365)
0.06
.11 (.10)
311 (/5859)
300 (78365)
.20
.18 (.16)
145 (/17849)
85 (/19966)
.13
.10 (.10)
2009.Q2
moderate + outsider
Freq.
Prox.
95 (/4019)
72 (/4856)
0.11
.09 (.01)
moderate + outsider
274 (/6408)
286 (/7255)
0.16
.11 (.10)
300 (/7158)
263 (/6888)
.20
.17 (.13)
2010.Q1
moderate + outsider
2010.Q3
2010.Q2
moderate + outsider
moderate + outsider
2009.Q3
2009.Q4
moderate + outsider
2009.Q1
moderate + outsider
99 (/25194)
77 (/25266)
.10
.12.(10)
Comment. In this table we used documents, i.e. single blog posts as contexts in order to compute associative relations from the word space. The table shows the total
amount of blog posts that contains the words moderat or social-democrat in parentheses, and the amount of blog entries that contains both the words moderat or
social-democrat + outsider. The proximity measures (which is the cosine angles between the context vectors) within parentheses are for the slang expression sossar,
which sometimes is used instead of Social Democrats.
In an attempt to get a more systematic comparison of the connection between

the terms related to the outsider concept in the blogosphere and the political parties, we counted the amount of blog entries where the party labels and the word
outsider are mentioned simultaneously. In connection to this saliency analysis
we also conducted an RI-analysis for the same time period. In this experiment,
we used documents (i.e., blog posts) as contexts in order to compute associative
relations, since we are interested in the extent to which the term outsider is associated with the two major parties. The results from this part of the analysis can be
found in Table 3.
Table 3 shows the number of blogs that contain the words Moderate or Social
Democrat together with the word outsider.Considering the particularly low proximity values, it is unclear if one can really draw any conclusions (since such low
similarity measure indicates that there is a lot of noise in the context vectors). For
this reason it may be better to simply compare the frequencies of the word outsider
in blog entries also containing the words Moderate and Social Democrat.
In general, the same number of blog entries occurs that contains the words
Moderate and Social Democrat, but the term outsider seems to occur more
frequently in messages containing the word Moderate. As mentioned above, the
results from the previous word space reveal that more specific terms were directly
related to concepts invented and used by the Conservative Moderate Party and its
alliance parties. Terms such as job-quake, contribution line, subsidy dependency,
tended to be more dominant in the blogosphere from late 2009 to mid 2010. A
cautious interpretation against this background would thus be that these results
together give some support to conclude that the language use of the Conservative
Moderate Party tends to be a bit more dominant in the blogosphere when it comes
to the concept of outsiders. In other words, the Conservative Moderate Party
seems to have been those that have set the scene for how one should define and
understand the concept of outsiders.
Summary and conclusions

The aim of this study has been to compare the language use in relation to the
targeted concept of outsiders in party related documents with the language use
in the Swedish blogosphere. The analysis of the party related documents from
2008 and onwards did, however, indicate a highly similar language use among the
Social Democratic- and the Conservative Moderate Party concerning the concept
of outsiders. The implication for our study in this respect was that we were not
able to identify any specific agenda setter in terms of language use, due to the
limitations in blogosphere data over time. Nevertheless, the results from the RI
87
88
analysis have validity. The top ten semantic similarities from the RI word space
model of the Swedish blogosphere were all well-known terms and concepts that
could be derived from the party related documents. From this perspective, we
have after all been able to identify how the term outsider has been framed by the
political parties and, hence, how this has contaminated the language use in the
blogosphere during late 2008 to 2010.
For example, in the beginning of the period from late 2008 to early 2009,
we were able to identify words related to housing situations and segregation in
connection to the outsider concept in the Swedish blogosphere. Talking about
outsiders in these terms was something that the Social Democratic Party tended
to do in the years before 2008 (the language use in the blogosphere seems to be
lagging behind in relation to the language used by the Social Democratic Party
during the investigated period). In the middle of the investigated time span, in the
period from late 2009 to mid 2010, the results from the RI word space revealed
specific terms in relation to outsiderness that were directly connected to concepts
invented and used by the Conservative Moderate Party and its alliance parties. We
also found that the term outsider tended to occur more frequently together with
the word Moderate than for Social Democrat in different blog entries. A cautious
interpretation against this background would thus be that the results all together
give some support for that the language use by the Conservative Moderate Party
tends to be a bit more indicative in the Swedish blogosphere when it comes to the
concept of outsiders, especially since the Social Democratic Party historically has
a tendency to adapt their language use in relation to the outsider concept to the
language used by the Conservative Moderate Party.
References
Chong, D. and N.J. Druckman. 2007. Framing public opinion in competitive democracies.
American Political Science Review 101(4), pp. 637655. DOI: 10.1017/S0003055407070554
Kanerva, P., J. Kristofersson and A. Holst. 2000. Random Indexing of Text Samples for Latent
Semantic Analysis. Paper presented at the Proceedings of the 22:nd Annual Conference of the
Cognitive Science Society.
Kinder, D.R. 2003. Communication and politics in the age of information. In D.O. Sears, L. Huddy
and R. Jervis, (eds.), Handbook of Political Psychology. Oxford: Oxford University Press.
Naurin, D., R. Eising, C. Mahoney, D. Coen, F. Baumgartner and S. Sauragger. 2009. A Systematic Comparative Research Project on Interest Group Politics in Europe. European Collaborative Research Projects in the Social Sciences (ECRP).
Oscarsson, H. and S. Holmberg. 2008. Regeringsskifte. Stockholm: Norstedts juridik.
Rueda, D. 2005. Insider-outsider politics in industrialized democracies: the challenge to social
democratic parties. American Political Science Review 99(1), pp. 6174. DOI: 10.1017/
S000305540505149X
Rueda, D. 2006. Social democracy and active labour-market policies: insiders, outsiders and the
politics of employment promotion. British Journal of Political Science 36(3), pp. 385406.
DOI: 10.1017/S0007123406000214
Sahlgren, M. 2006. The Word-Space Model: Using Distributional Analysis to Represent Syntagmatic and Paradigmatic Relations between Words in High-Dimensional Vector Spaces.
Stockholm: Stockholm University.
Sahlgren, M. 2005. An introduction to random indexing. Proceedings of the Methods and Applications of Semantic Indexing Workshop at the 7th International Conference on Terminology
and Knowledge Engineering, TKE 2005, August 16, Copenhagen, Denmark.
Schtze, H. 1993. Word space. In Proceedings of the 1993 Conference on Advances in Neural Information Processing Systems. San Francisco: Morgan Kaufmann Publishers Inc.,
pp. 895902.
Slothuus, R. 2008. How Political Elites Influence Public Opinion: Psychological and Contextual
Conditions of Framing Effects. rhus: Politica.
Slothuus, R. and C. de Vreese. (2008). Political parties, motivated reasoning, and issue framing
effects. Unpublished manuscript, Amsterdam.
Turney, P. and P. Pantel. 2010. From frequency to meaning: vector space models of semantics.
Journal of Artificial Intelligence Research (JAIR) 37(1), pp. 141188. AI Access Foundation.
Walgrave, S. and K. De Swert. 2007. Where does issue ownership come from? From the party
or from the media? Issue-party identifications in Belgium, 19912005. The International
Journal of Press/Politics 12(1), pp. 3767.
Appendix
(1)
(2)
(3)
(4)
(5)
(6)
Mlet med frndringen av systemet r att hjlpa mnniskor tillbaka i arbete och minska
gapet mellan utanfrskap och arbete.
Vi vljer att sl vakt om Sveriges sunda offentliga finanser. Vi vljer att fra en tydlig
politik fr full sysselsttning och minskat utanfrskap.
Under Socialdemokraternas tid i regeringen kade sjukskrivningarna kraftigt. Fr att
frbttra statistiken frde man ver 140 personer om dagen till frtidspension och ett
permanent utanfrskap.
Det ndvndigt att se vad som ligger bakom brottsligheten: ofta rr det sig om brist p
jobb, brist p sociala gemenskaper och brist p trygghet. Drfr behver vi arbeta nnu
mer fr att bryta utanfrskapet: fr fler i arbete, fr nya arbetstillfllen, fr en bttre
skola och fr att nyanlnda ska integreras. P s stt fr fler barn en trygg uppvxt den
mest effektiva brottsfrebyggande insatsen. Genom att arbeta mot utanfrskap arbetar
vi ocks mot brottsligheten, detta r ett samband s starkt att det ibland glms bort.
Brottsligheten bekmpas genom ett starkt rttsvsende och genom en politik som minskar utanfrskap och arbetslshet.
Ett stort antal i utanfrskap hotar vlfrden eftersom frre bidra till att skapa de begrnsade resurser som mste delas av fler mnniskor. Regeringens anstrngningar fr
hg sysselsttning r det enda lngsiktigt hllbara sttet att skydda vlfrden. ( --- )
De tgrder som har vidtagits fr att minska utanfrskapet bidrar ocks till att skra
pensionerna.
89
(7)
Vad r svaret frn regeringen Reinfeldt idag? Utanfrskapet kade i juni fr tredje
mnaden i rad.

( --- ) Det har gtt en halv mandatperiod. Det r dags fr utvrderingen. Det r upp till
bevis fr regeringen.

Gjorde det vad de sa att de skulle gra? Hur gick det med jobben? Hur gick det med
utanfrskapet? Hur gick det med vlfrden och tryggheten? Vad r regeringen Reinfeldts
besked idag?
(8) Sysselsttningen behver ka bland unga, bland invandrade svenskar, bland deltidsarbetande kvinnor och, inte minst, bland ldre. Ska vi n dit mste den ofrihet som stavas
arbetslshet och utanfrskap bekmpas och fr den delen ofrivillig deltidsarbetslshet.
Ska vi n dit mste arbetslinjen glla alla.
(9) Under det senaste ret har antalet arbetslsa kat med 100 000. Sedan valet 2006 har
utanfrskapet vuxit med 70 000 mnniskor,. ( --- ) I valet 2010 kommer arbetslsheten
att vara den allt annat verskuggande frgan.
(10) Fr snart fyra r sedan vann de borgerliga partierna valet p att utlova fler jobb och frre
i utanfrskap. Nr vi nu kan utvrdera resultatet av mandatperioden r det dessvrre
nedslende. Regeringen kastar ut tiotusentals svenska medborgare ur sjukfrskringen
in i frskringslshet. P bara ett r har antalet arbetslsa kat med fler n 100000 personer, och antalet i utanfrskap med 70 000 personer. Utbetalningarna av socialbidrag
berknas ka med 50 procent mellan 2006 och 2011.
(11) Idag kom siffror frn SCB som visar att utanfrskapet, Moderaternas eget mtt p hur vl
politiken fungerar, har kat med ca 70 000 mellan 2006 och 2009. Igr kom siffror som
visar att socialbidragen, den allvarligaste formen av utanfrskap, kat i ver 90 procent
av kommunerna frra ret.
Table 2. Words related to the term outsider in the Swedish blogosphere between 2008 to 2010 (in Swedish).
2008.Q4
Utanfrskap:
Prox.
2009.Q1
Utanfrskap:
Prox.
2009.Q2
Utanfrskap:
Prox.
2009.Q3
Utanfrskap:
Prox.
grogrund:
otrygghet:
utanfrskapet:
mervrden:
skyddsnt:
trngboddhet:
ntverk:
frutsttningar:
rdsla:
frkttrade:
0.25
0.24
0.22
0.20
0.19
0.18
0.18
0.18
0.18
0.18
lngtidsarbetsls:
inskriven:
frmtts:
trngboddhet:
omhndertagit:
bostadsker:
kontaktats:
fattigdom:
kriminalitet:
frutsgelser:
0.34
0.34
0.22
0.21
0.20
0.15
0.15
0.15
0.14
0.14
bidragsberoende:
bidragslinjen:
segregation:
fattigdom:
arbetslshet:
vlstnd:
bostadsbrist:
arbetskraftsbrist:
bostadslshet:
otrygghet:
0.39
0.33
0.30
0.26
0.25
0.24
0.23
0.23
0.23
0.22
fattigdom:
jobbvning:
regelkrngel:
strukturell:
elevantal:
omfng:
bostadsbrist:
segregation:
otrygghet:
inkomstklyftor:
0.33
0.31
0.28
0.26
0.25
0.23
0.23
0.23
0.21
0.20
2009.Q4
Utanfrskap:
Prox.
2010.Q1
Utanfrskap:
Prox.
2010.Q2
Utanfrskap:
Prox.
2010.Q3
Uanfrskap:
Prox.
jobbvning:
arbetslshet:
bidragsberoende:
sjukfrskring:
frre:
fattigdom:
utsatthet:
trygghet:
alliansen:
missbruk:
0.35
0.30
0.28
0.25
0.25
0.22
0.22
0.22
0.21
0.21
lngtidsarbetslshet:
alliansen:
arbetslshet:
skymundan:
vlfrd:
segregation:
moderatledda:
frre:
jobbvning:
regeringen:
0.25
0.25
0.22
0.21
0.21
0.20
0.20
0.20
0.20
0.19
bidragsberoende:
smnbehov :
konsumtionsutrymme:
socialbidragsberoende:
risktagande:
fattigdom:
segregation:
behov :
inflytande:
energiintag:
0.33
0.31
0.28
0.26
0.26
0.26
0.26
0.25
0.24
0.24
bidragsberoende:
fattigdom:
smnbehov :
omstllningsfrskring:
arbetsutbud:
otrygghet:
kriminalitet:
medborgarinflytande:
markant:
nrpoliser:
0.43
0.30
0.24
0.24
0.24
0.23
0.23
0.22
0.22
0.22
91
chapter 5
Text to ideology or text to party status?*

Graeme Hirst, Yaroslav Riabinin, Jory Graham,
Magali Boizot-Roche, and Colin Morris
Department of Computer Science, University of Toronto
Several recent papers have used support-vector machines with word features to
classify political texts in particular, legislative speech by ideology. Our own
work on this topic led us to hypothesize that such classifiers are sensitive not to
expressions of ideology but rather to expressions of attack and defence, opposition and government. We tested this hypothesis by training on one parliament
and testing on another in which party roles have been interchanged, and we find
that the performance of the classifier completely disintegrates. But removing the
factor of governmentopposition status, as in the European Parliament, enables
a more-ideological classification. Our results suggest that the language of attack
and defence, of government and opposition, may dominate and confound any
sensitivity to ideology in these kinds of classifiers.
1. Introduction
There have been a number of attempts recently to develop methods to automatically determine the ideological position of a political text. For example, one might
wish to take a newspaper editorial or a blog and classify it as socialist, conservative, or Green. In practice, much of the research has taken speeches by members
of a legislature (such as the U.S. Congress or the European Parliament) as the text
to be classified and indicators such as party membership or legislative voting patterns as a proxy for ideology (indeed, Yu et al. (2008) use the terms party classifier
and ideology classifier almost interchangeably); thus the problem becomes one of
predicting one of these indicators from speech. One might expect, a priori, that
* This is an extended version of Party status as a confound in the automatic classification
of political text by Graeme Hirst, Yaroslav Riabinin, and Jory Graham, Proceedings, 10th
International Conference on Statistical Analysis of Textual Data (JADT 2010), Rome, June 2010.
This work is financially supported by the Natural Sciences and Engineering Research Council
of Canada.
94 Graeme Hirst et al.
methods based solely on the vocabulary used in a text would not be effective,
because the members of a legislature, regardless of ideology, are all discussing the
same topics e.g., the legislation before them or the issues of the day and hence
would all be using the same topic-derived vocabulary (Mullen and Malouf 2006).
The ideology expressed in a text would thus be apparent only at the sentenceand text-meaning levels. Nonetheless, one might hypothesize that different ideological frameworks lead to sufficiently different ways of talking about a topic that
vocabulary can be a discriminating feature (Lin et al. 2006). And indeed, several
studies have obtained notable results merely from classification by support-vector
machines (SVMs) with words as features (bag-of-words classification).1
For example, Thomas et al. (2006) examined speeches made by members of
the U.S. House of Representatives to try to determine whether each speaker supported or opposed the proposed legislation under discussion. They combined bagof-words text classification by SVMs with textual information about each speakers
agreement or disagreement with other speakers, obtaining an accuracy of around
70% (the majority baseline was 58%). Greene (2007) obtained an improved accuracy of over 74% on the same task by annotating each word with its grammatical
relation from a dependency parse. Jiang and Argamon (2008), on the related task
of classifying political blogs as liberal or conservative, improved results over using
word features of the whole text by first trying to identify subjective sentences and
the expressions of opinion that they contain, and then limiting the features to
those parts of the text.
Diermeier et al. (2007) used SVMs with bag-of-words features to classify
members of the U.S. Senate by ideology, labelling each speaker as a liberal or a
conservative, and achieved up to 94% accuracy. However, in these experiments,
the authors focused on extreme senators the 25 most conservative and the 25
most liberal members in each Senate. On moderate senators, the results were
notably poorer (as low as 52% accuracy). Moreover, there was considerable overlap between the training and testing portions of Diermeier et al.s dataset, since
they extracted content from multiple Senates (101st to 108th) and since members of Congress tend to preserve their beliefs over time. Specifically, 44 of the 50
extreme Senators in their test set were also represented in the training data, which
means that the classifier was already trained on speeches made by these particular
individuals. Thus the classifier might be learning to discern speaking styles rather
than ideological perspectives.
1. Observe that this goal differs from that of, e.g., Gryc and Moilanen (this volume) and Dahlberg
and Salhgren (this volume), who aim to determine the position expressed in a text with regard to
a particular topic, such as Barack Obama or outsiders in Sweden. By contrast, the more general
goal here is the ideological position underlying a text, independent of any particular topic.
Chapter 5. Text to ideology or text to party status?
Later work by the same authors (Yu et al. 2008) made no distinction between
moderates and extremes; rather, they tried to classify all members of the 2005
U.S. Congress by party affiliation, achieving an accuracy of 80.1% on the House of
Representatives and 86.0% on the Senate. The goal of their study was to examine
the person- and time-dependency of the classifier by using speeches from both
the Senate and the House and comparing the results. They found that party classifiers trained on House speeches could be generalized to Senate speeches of the
same year, but not vice versa. They also observed that classifiers trained on House
speeches performed better on Senate speeches from recent years than older ones,
which indicates the classifiers time-dependency.
We began the present work to see whether these kinds of bag-of-words SVM
classification methods would hold up in analysis of speech in the Canadian
Parliament (Section3 below). Our results, however, led us to question whether
vocabulary differences between parties really reflected ideology or whether they
had more to do with each partys role in the Parliament, and we investigate this
in Section4 below.
2. Background: The Canadian party system and Parliament
The Canadian Parliament is a Westminster-style parliament. The party with the
most seats in the House of Commons (albeit possibly a minority of them) forms
the government; the other parties are the opposition. There may also be a few
Independent (unaffiliated) members. In the last 12 years, there have been four
or five parties in each Parliament. In broad terms the parties may be classified as
conservative (Reform Party, Canadian Reform Conservative Alliance, Progressive
Conservative Party, Conservative Party of Canada), liberal or centre (Liberal
Party),2 or left-wing (New Democratic Party and Bloc Qubcois); see Collette
and Ptry (this volume) for more discussion of the parties leftright positions.
Both English and French are official languages of Canada. A speaker in
Parliament may use either language, and will sometimes even switch between the
two within a speech. Everything said in Parliament is professionally translated into
the other official language, and the proceedings are published in both languages.
Thus the published English text of the debates is a mixture of original English and
translations from French, and the French text has the complementary distribution.
2. Thus in our data, all liberals are Liberals, but not all conservatives are Conservatives. Similarly,
we distinguish between opposition parties any party that is not the governing partyand the
Opposition party the opposition party from which the Leader of the Opposition is drawn.
95
3. First set of experiments: Classifying by party

The present work was intended as a prelude to a larger project on ideological
analysis of text. Our first task, intended as a baseline, was to apply bag-of-words
support-vector machine classification, as used by Diermeier et al. (2007) and Yu
et al. (2008) on U.S. Congressional speech, to speech in the Canadian Parliament,
to see whether we could classify the speech by party affiliation (as a proxy for ideology) and obtain similar results, despite the differences in the political systems
of the two countries.
In Canadian politics, unlike those of the U.S., party discipline is strong
and (with only rare exceptions) all members of a party will vote the same way.
The governing party will always vote to support its legislation; an opposition
party might oppose it or support it. Thus (in contrast to the tasks described by
Diermeier et al. and Yu et al.), there is no meaningful distinction between predicting voting records from parliamentary speech and predicting party affiliation. On one hand, it might be argued that this makes the task easier because
parliamentary speech is likely to be highly partisan. On the other hand, it might
be argued that it makes the task more difficult, because there is a greater diversity
of views with precisely the same voting pattern, and so the classification is less
straightforward.
In order to avoid the problems inherent in cross-time analysis, as highlighted
by the work of Diermeier et al. (2007), we focus in this section on a single time
period, so that there is a one-to-one mapping between members of Parliament
(MPs) and documents in our dataset. Each document is a concatenation of all the
speeches made by a speaker, and no other document contains text spoken by that
person. Thus no speaker appears in both training and test data.
3.1
Data
We used both the English and French House of Commons Debates (Hansard)
for the first 350 sitting days of the 36th Parliament (1997-09-22 to 2000-05-10).
In the 36th Parliament, a majority government was formed by the Liberal Party,
led by Jean Chrtien. This data was available in a convenient plain-text form with
sentence breaks identified (Germann 2001), as it has been widely used for research
in machine translation.
We considered two sections of the proceedings: the debates on legislation and
other statements by members (Government Orders) and the oral question period.
And we focused on the governing Liberal Party and the opposition conservative
parties,3 in order to do a binary discrimination, liberal versus conservative; the

left-wing parties had relatively few members in this Parliament and were excluded
from the analysis.
For each MP who was a member of one of the liberal or conservative parties, and for each language, we formed a document by concatenating all their
utterances in debates, question period, or both, throughout the Parliament. (For
simplicity, we will refer to all utterances as speeches, regardless of their length,
including questions and answers in the oral question period.) We experimented
with a variety of pre-processing methods, including stemming the words or
leaving them whole, removing or retaining stopwords (defined as the 500 most
frequent words in the text), and removing or retaining rare words (defined as
those occurring in fewer than five documents). (Details of these and other preprocessing matters are given by Riabinin 2009.) In some of our experiments, we
discarded the data for members who said very little, or nothing at all, in question
period or in debates, using 200 documents representing 121 liberals and 79 conservatives; in other experiments, we considered all 156 liberal and 79 conservative
members who spoke at all.4 In all, depending on our choices in pre-processing, we
had about 4 million words in each language for liberals (of which approximately
900,000 were from the question periods) and 2.7 million for conservatives (of
which approximately 500,000 were from the question periods).
Generally, these variations in pre-processing made little difference to the
results. In this paper we report results for experiments on the texts for all speakers,
with words left unstemmed and with rare words removed, which usually, though
not invariably, gave the best results.
3.2
Method
Taking word-types as the features for classification that is, regarding the document for each speaker as a bag of words for each language we trained an SVM
classifier for ideology as indicated by party membership, liberal or conservative.
3. At the time of this Parliament, the conservative parties were in disarray. The Opposition was
the conservative Reform Party (which became the Canadian Reform Conservative Alliance in
March 2000), but the conservative Progressive Conservative Party also held a number of seats.
4. Several members of the conservative parties either defected to the Liberal party or became
independents during this Parliament; and one member of the left-wing NDP defected to a
conservative party. We treated all these members as conservatives in our experiments; for details
and rationale, see Riabinin (2009).
97
In training and testing, we used five-fold cross-validation. We experimented with

four weighting schemes: boolean (presence of feature), tf (term frequency), tf-norm
(term frequency normalized by document length), and tf-idf (term frequency by
inverse document frequency). The best results were obtained with tf-norm and
tf-idf; the results we present below all use the latter.
3.3
Results
Table 1 shows the accuracy of classification of party membership by the SVM for
each language on the documents of each data set: oral question period (OQP),
debates (GOV), and the two combined (OQP+GOV). In all cases, retaining the
500 most frequent features led to higher accuracy than removing them. The baseline method of choosing the larger class (liberal) for all members would give an
accuracy of 65.5%. All our results are well above this baseline, and in fact reach
almost 97% for oral question period in English when frequent words are retained.
The reason for the discrepancy between this result and the 89.5% obtained for the
same data in French is unclear, as the two texts are mutual translations and no
such effect was seen with the debates texts.5 We also observe that in three cases
out of four, combining debates and question period in a single classifier is deleterious to accuracy compared to classifying each separately. Generally speaking, our
results are similar to, or better than, those of Yu et al. (2008) on the U.S. Congress.
Table 1. Accuracy (%) of classification by ideology on speech in the oral question period
(OQP) and debates (GOV) by liberal and conservative members of the 36th Parliament,
with and without removal of the 500 most frequent features (majority baseline = 65.5%).
With 500 most frequent features retained
English
French
With 500 most frequent features removed
English
French
OQP+GOV
OQP
GOV
83.8
83.2
96.9
89.5
83.3
86.0
78.7
80.8
92.9
84.8
79.6
83.5
5. Compare the results of Collette and Ptry (this volume) on the differences that they found
in locating English and French political manifestos on a leftright spectrum, and the differences
between languages that they adduce in explanation. In our results, however, while the accuracy
obtained for each language sometimes varies quite noticeably within each condition, there is
no apparent system in the differences; sometimes the English results are more accurate and
sometimes the French results are; sometimes the difference is marginal.
3.4
Discussion
The higher accuracy of classification for question period than for debates suggests
that the language of question period is in some way more partisan than that of
debates. However, our examination of the most discriminative words suggests that
this partisanship is not so much ideological as a matter of attack and defence. In
particular, in the Canadian Parliament, the oral question period consists largely of
hostile questions from members of the opposition parties to ministers of the government, with only occasional friendly questions from government backbenchers,
which themselves often serve primarily to set up an attack on the opposition.6 Its
possible, therefore, that our classifier may be learning at least in part not to
distinguish ideologies but to distinguish questions from answers or attack from
defence, which is not the goal of our research. Table 2 shows the ten most discriminative English words for each side in question period. For the governing liberals,
the top words are hon and member, as in the hon. member for Halifax West, which
is how a minister from the governing party typically addresses a member who has
asked a question. Also, the word we might be used by a minister to speak on behalf
of the entire party or government when responding to questions. For the opposition
conservatives, the word why serves the obvious purpose of posing a question, and
the words he and her are likely used to refer to government ministers who are the
targets of the questioning. Also, observe the use of words such as bloc, reform, and
opposite by the liberals, and prime (as in Prime Minister) and liberal and liberals by
the conservatives.7 This lends further support to the hypothesis that the classifier is
partially learning to distinguish government members from opposition members.
When frequent words are removed we see this effect less, with a corresponding
drop in accuracy (see the second part of Table 1), but it does not disappear entirely.
In this condition, we certainly see reflections of ideology in vocabulary. The liberal lexicon is characterized by words related to Qubec (French, Francophonie,
MAI [Montral Arts Interculturels], PQ [Parti Qubcois]) and various social issues
(housing, violence, humanitarian, youth, society, technology), while the conservatives tend to focus on monetary concerns (APEC, taxpayer, dollar, millions, paying,
premiums), aboriginal affairs (native, Indian, chief), and, to a lesser degree, national
defence (military, marshall). Nonetheless, the governing liberals use language that
6. This contrasts with the practice in similar parliaments, such as those of Australia and the
U.K., in which questions are more evenly balanced between those of the opposition and those
of government backbenchers.
7. Interestingly, this tendency for the names of opponents to be discriminating features is the
converse of what Lin et al. (2006) found in their analysis of an IsraeliPalestinian debate, in
which naming ones own side was discriminating; but see Section6.4.3 below.
99
Table 2. The top 10 English words characterizing each class in the oral question period.
Rank
liberal
(government)
conservative
(opposition)
1
2
3
4
5
6
7
8
9
10
hon
member
we
opposite
quebec
housing
bloc
reform
qubcois
women
prime
why
liberal
solicitor
farmers
finance
he
liberals
hrdca
banks
a HRDC
= Human Resources Development Canada, a federal government department.
is generally positive (congratulate, excellent, progress) and is intended to create the

appearance of a government at work (established, inform, improve, assist, developing, promote). In contrast, the opposition conservatives use negative words that
are meant to call the governments competence into question (justify, resign, failed,
admit, refusing, mismanage). So again, it seems that many of the features relate not
to ideology but to attack and defence not to the partys beliefs but to its status as
government or opposition.
4. Second set of experiments: Classifying by party status
Even if a classifier for political speech were truly using features related to ideology,
we would expect that at least some of these features would specifically pertain to
views of current events and therefore, if it is trained on one Parliament, it will not
perform as well on a different Parliament in which different events are current,
as in the results of Diermeier et al. discussed in Section1 above. Nonetheless, we
would expect that many of the features will be invariant over time and that such a
classifier will still perform much better than a baseline.
On the other hand, if the ideological classifier is in reality using (solely or
primarily) features related to government and opposition status, then training on
one Parliament would carry over only to other Parliaments in which the parties
hold the same status; if they swap roles, then the classifier will fail. Indeed, in such
a case it might (or should!) perform worse than the majority baseline, tending to
classify liberals as conservatives and vice versa. In our second set of experiments,
we tested the hypothesis that the latter is the case that an SVM bag-of-words
Chapter 5. Text to ideology or text to party status? 101
classifier for Canadian parliamentary speech is primarily sensitive to party status,

not ideology. We also looked at the in-between case: training an ideological classifier on data in which all combinations of ideology and party status are present.
4.1
Data
To test our hypothesis, we needed a Parliament in which, in contrast to the 36th

Parliament, a conservative party was in government. We chose the recent 39th
Parliament (2006-04-03 to 2008-09-07), with a minority Conservative Party8,9
government led by Stephen Harper; the Liberal Party was in opposition, along
with the New Democratic Party and the Bloc Qubcois. The proceedings were
downloaded from the Parliament of Canada website in HTML-formatted documents and processed into a format similar to that of the 36th Parliament data.
4.2
Method and results
4.2.1 Replication of the first experiments on the new data

We first replicated the experiments of Section3 on the new data, discriminating
liberal members from conservative members (there was sufficient data for 104
liberals and 130 conservatives) within the same Parliament. Training and testing
with five-fold cross-validation on the 39th Parliament, we achieved results similar
to those of the 36th Parliament, albeit with slightly lower accuracy, especially for
the English OQP documents; see Table 3 and compare Table 1. In particular, the
accuracy of the classification on French text of speakers in Government Orders is
anomalously low (baseline level) compared to all our other results including those
for the English translation of the same text; we have no explanation for this. We
also observe that for this data, unlike the 36th Parliament, the strategy of removing the 500 most frequent words is sometimes superior to that of retaining them.
Examining the primary features used in the classification for oral question
periods, we observed that several words swapped sides: four of the top 10 English
words that characterized the liberals in the 36th Parliament characterized conservatives in the 39th Parliament, and the primary word that characterized conservatives in the 36th Parliament was the second word that characterized liberals
in the 39th; see Table 4. This is evidence for our hypothesis that the classifier is
really picking up features related to government and opposition status.
8. So in this Parliament, unlike the 36th, all conservatives are Conservatives.

9. http://www2.parl.gc.ca/housechamberbusiness/ChamberSittings.aspx
Table 3. Accuracy (%) of classification by ideology on the 39th Parliament,

with and without the 500 most frequent words retained (majority baseline = 55.8%).
English
French
English
French
OQP+GOV
OQP
GOV
83.8
75.5
88.3
88.8
72.3
56.8
79.9
79.0
83.5
88.2
73.2
57.2
Table 4. The top 10 English words characterizing each class in oral question periods in
each Parliament (extending Table 2). Boldface indicates words that swap sides between
the two Parliaments. Boldface italic words characterize the governing side; the boldface
roman word characterizes the opposition.
Rank
1
2
3
4
5
6
7
8
9
10
36th Parliament
39th Parliament
liberal
(government)
conservative
(opposition)
liberal
(opposition)
conservative
(government)
hon
member
we
opposite
quebec
housing
bloc
reform
qubcois
women
prime
why
liberal
solicitor
farmers
finance
he
liberals
hrdc
banks
conservatives
prime
conservative
immigration
mulroney
kyoto
admit
minority
promise
her
bloc
liberals
senate
violent
we
qubcois
greenhouse
ndp
corruption
member
4.2.2 Classifying across Parliaments

Again we used the proceedings of the 36th and 39th Parliaments, both English
and French, but in each language we took the classifiers trained on one Parliament
and tested them on the other. (In these experiments, we have the deprecated situation that some individual speakers, being members of both parliaments, occur
in both the training data and the test data and thereby might give the classifier an
unfair boost.) The results, shown in Table 5, are in all cases well below the majority baseline scores, just as we hypothesized; when party status changes, there are
no constant ideological features to save the classifier.
We also tried training classifiers on the data of the two Parliaments combined. This dataset includes all combinations of ideology and party status that
is liberals in government, liberals in opposition, conservatives in government,
Table 5. Accuracy (%) of classification by ideology when training on one Parliament

(36th or 39th) and testing on the other.
Training Testing
36 39 (Majority baseline = 55.8%)
English
French
English
French
39 36 (Majority baseline = 65.5%)
English
French
English
French
OQP+GOV
OQP
GOV
44.9
45.7
43.3
46.1
44.6
47.0
46.2
43.5
44.6
49.6
44.1
43.5
36.8
35.2
34.5
51.1
36.2
33.5
35.0
36.4
49.6
51.1
42.7
33.5
and conservatives in opposition. Some speakers, those who were members of

both Parliaments, appear with each possible party status, whereas others, those
who were members of only one of the two Parliaments, appear in only one of
these four conditions. A classifier trained on the former group performs at
around the level of the majority baseline (Table 6); one trained on the latter
does better (Table 7), but the results are overall below the level of the original
experiments (Tables 1 and 3), especially for OQP data. (The exception is that
the anomalously low results for French GOV data are not seen when frequent
features are retained.)
Table 6. Accuracy (%) of classification by ideology on speakers who were members
of both the 36th (liberal government) and 39th Parliament (conservative government),
with and without the 500 most frequent words retained (majority baseline = 64.0%).
English
French
English
French
OQP+GOV
OQP
GOV
62.0
63.0
66.9
63.0
61.1
63.0
64.0
64.0
66.9
64.0
59.4
64.0
Table 7. Accuracy (%) of classification by ideology on speakers who were members of

either the 36th (liberal government) or 39th Parliament (conservative government), but not
both, with and without the 500 most frequent words retained (majority baseline = 51.9%).
English
French
English
French
OQP+GOV
OQP
GOV
78.5
76.6
81.7
78.3
72.6
71.2
76.3
75.0
73.5
76.1
71.9
61.9
4.2.3 Including the other opposition parties

Another way to see whether the classifier is more sensitive to party status than
to ideology is to muddy the ideological waters by including the left-wing parties,
which were in opposition in both Parliaments, in the analysis. If the classification were truly ideological, lumping these parties in with the other conservative
(36th Parliament) or liberal (39th Parliament) opposition parties would markedly
degrade the performance of the classifier. On the other hand, if party status is what
matters, there should be little effect in doing so as the opposition parties will be
more or less indistinguishable. We carried out this experiment on the English data
with frequent words retained.
The results are shown in Table 8. They should be compared with the liberal/
conservative results for the same Parliament and same processing method, shown
in the first lines of Table 1 (96.9%, 83.3%) and Table 3 (88.3%, 72.3%). There
is almost no degradation of performance on the 36th Parliament; for the 39th
Parliament, there is a noticeable drop (10.12 percentage points) for the question
period, but little for the debates.
Table 8. Accuracy (%) of classification of government and opposition (all parties)
on English text of the 36th and 39th Parliaments with the 500 most frequent words
retained (majority baselines = 51.5% and 59.4% respectively).
36th
39th
OQP
GOV
95.6
78.2
82.6
70.9
4.3
Discussion
The results seen in Sections4.2.13 are consistent with the hypothesis that the
SVM bag-of-words classifier is sensitive not to expressions of ideology for which
party membership is a reasonable proxy, but rather to expressions of attack and
defence, opposition and government. When we train on one parliament and test
on another in which party roles have been interchanged, the performance of
the classifier completely disintegrates; the degradation is far worse than can be
explained merely by the difference between the two parliaments in the vocabulary of the current topics of discussion. Some features that are indicative of each
party swap sides with the change of government. And combining ideologically
inconsistent opposition parties in the classifier does not in most cases seriously
degrade its performance.
5. Classification based on the emotional content of speeches
Recall that our feature analysis of the 36th Parliament showed that liberal members tended to use words that convey a more positive sentiment than those used
by conservatives. This suggests that it might be possible to distinguish parties or
ideologies (solely) by the emotional content of their speeches. Indeed, researchers
such as James Pennebaker have made something of an industry of interpreting
politicians from a statistical analysis of their use of a single category of words. For
example, during the 2008 U.S. presidential election, Pennebaker (2008) wrote:
Over the last few years, some have argued that the use of negations (e.g., no, not,
never) indicate [sic] a sign of inhibition or constraint. Low use of negations may
be linked to impulsiveness. Across the election cycle, Obama has consistently
been the highest user of negations suggesting a restrained approach where as
[sic] McCain has been the lowest a more impulsive way of dealing with the world.
Similarly, Pennebaker concluded that McCains greater use than Obama of the
first-person singular (I, me, my) signalled a likely greater openness and honesty.10
In the context of our results above, the questions we ask are not just whether
liberals can be distinguished from conservatives in the Canadian Parliament
10. The validity of this kind of analysis is discussed and defended by Pennebaker et al. (2007a).
But Pennebaker (2008) also writes: No one should take any text analysis experts opinions too
seriously. The art of computer-based language analysis is in its infancy. We are better than tealeaf readers but probably not much.
merely by the emotional content of their speeches, but also, if so, whether the
feature actually discriminates ideology (in line with the stereotype of happy liberals, dour conservatives) or is again confounded by the parties status in the
Parliament.
5.1
Method and data
To test these questions, we used Pennebaker et al.s (2007b) software Linguistic

Inquiry and Word Count (LIWC2007). LIWC counts the proportion in a text of
particular words and word stems in over 60 categories, including linguistic properties (pronouns, adverbs, prepositions, etc), psychological denotation (positive
emotion, negative emotion, etc), and various topics (work, money, religion, etc);
it does not, itself, provide any interpretation of the counts.
For these experiments, we used the English speeches of the oral question
periods and debates of the 36th and 39th Parliaments, excluding MPs who spoke
very little. This gave us a dataset of documents for 200 MPs (121 liberals, 79 conservatives) in the 36th Parliament and 220 MPs (125 conservatives and 95 liberals) in the 39th Parliament. First, we ran LIWC on this data, which gave us a
64-component vector for each document, each component being the value that
LIWC computed for the document for one of its categories. We then performed
classification experiments on the data (with five-fold cross-validation) using this
64-component representation of the documents, in order to see whether positive
and negative emotion were among the top discriminating features for liberals and
conservatives, respectively. Then we repeated the classification, using only positive
emotion and negative emotion (referred to as posemo and negemo) as features.
Finally, we performed a third experiment, in which affect was reduced to a single feature, the amount by which the positive emotion in the text exceeded the
negative (i.e., posemo minus negemo); this representation does not distinguish a
completely unemotional text from one that contains emotion of each polarity in
equal amounts.
5.2
Results
Table 9 shows the results of these experiments. In the first experiment, with 64
features, the accuracy for both datasets was equal to the majority baseline, because
all MPs were classified as members of the majority party! In contrast, using only
posemo and negemo, either as two features or as a single feature, yielded a substantial improvement of up to 20.5 percentage points over the baseline (a relative
Table 9. Accuracy (%) of classification by party using LIWC features for Englishtext
of the 36th and 39th Parliaments oral question period (OQP) and debates (GOV)
(majority baseline = 60.5% and 56.8% respectively).
36th
64 features
posemo and negemo
posemo minus negemo
39th
OQP
GOV
OQP
GOV
60.5
80.5
81.0
60.5
79.5
78.5
56.8
73.1
72.2
56.8
55.0
59.1
error reduction of 51.9%) for the 36th Parliament and 16.3 points for the oral
question periods of the 39th. However, performance remained around baseline
for the debates of the 39th Parliament.
Nonetheless, a feature analysis confirmed that in the 36th Parliament, positive emotion was among the top five liberal features and negative emotion was
among the top ten conservative features, whereas in the 39th Parliament, positive
emotion was the fourth feature for conservatives in oral question periods and
sixth in debates, whereas negative emotion was eighth and tenth respectively for
liberals. Hence, we can see that positive emotion is a characteristic of members
of the governing party, and negative emotion is a characteristic of members of
an opposition party; again, party status confounds ideological classification. The
result of the classifier on all 64 features may be explained by the fact that no
LIWC category had a significant impact on the classification. In other words,
even though some LIWC categories were discriminating features for liberals
and others were discriminating features for conservatives, the overall difference
between the two groups was so slight that without feature selection the resulting
classifier simply labelled all test instances as belonging to the majority class. This
seems to be the case also for posemo and negemo by themselves in debates in
the 39th Parliament.
6. Third set of experiments: European Parliamentary data
If our ideological classifier is in reality sensitive to government and opposition,
then this effect should disappear when it is applied to data in which there is no
government or opposition per se, but merely position-based debate with a more
or less equal amount of attack and defence on both sides. Such a situation may
be found in the European Parliament, in which a leftright ideological division
dominates governmentopposition divisions (Hix et al. 2007).
Our goal here is thus very similar to one of the tasks of the 2009 DEFT text
mining challenge11 (DEFT 09): classification by political group of speeches by
Members of the European Parliament (MEPs). The DEFT corpus consisted of
speeches from 1999 to 2004 by MEPs belonging to the five largest groups. Three
teams attempted this task, but two declined to share their results. The remaining
team, from the University of Montreal (Forest et al. 2009), reported an F-measure
of about 0.33 on multiclass classification, which the organizers described as mediocre as the random baseline accuracy for the corpus was about 28% (Grouin et al.
2009:49). We attempted both binary classification of left-wing versus right-wing
MEPs, and multiclass classification of MEPs from the five largest groups, as in the
DEFT task.
6.1
Data
We used English data from the proceedings of the European Parliament as our
corpus.12 Ranging from 2000 to early 2010, it was almost a strict superset of
that used in the DEFT task. However, the data used in the DEFT task had been
stripped of any explicit references to groups. Thus, tokens such as PPE, ChristianDemocrat, and United Left, were all replaced with an anonymous tag. We understand that this was because, in the DEFT task, the organizers had human judges
attempt a classification on the same data for comparison, and phrases such as As
vice-chairman of the PPE-DE group, I were presumably considered too much of
a giveaway to a human reader. By contrast, we left all group names in place in our
data. In Section6.4.3 below, we will discuss the effect that anonymization has on
classification.
6.2 Method
The choice of how to organize the raw text into vectors proved to be a key one.
Our first approach was, for each MEP, to concatenate all of their utterances and
consider that to be one document, as we did for the Canadian Parliament. This
11. DEFT (DEfi Fouille de Textes) is an annual challenge and evaluation conference for researchers in text mining and classification. Each year, one or more tasks related to text mining are
set, and training and test corpora are provided; research teams compete to get the best results.
Results and methods are then discussed at the conference.
12. The data was collected and marked-up in XML by Dr Maarten Marx of the University of
Amsterdam, who kindly made it available to us; see Marx and Schuth (2010) for details of the
XML markup.
contrasted with the approach taken in the DEFT challenge, in which each individual speech remained a separate document. With our concatenation policy,
however, we achieved accuracy only slightly above a random-guessing baseline.
However, we observed that the amount of text we had per MEP varied widely,
from a few hundred words to tens of thousands of words. Yet each MEPs document was being turned into a vector that affected the classifier equally, contradicting the natural notion that a document should have an affect on the classifier
commensurate with its size. We rectified this by dividing each MEPs concatenated
utterances into a number of equal-sized documents. We experimented with different document sizes, and found that it had a marked effect on accuracy (as shown
in our results below).13 The sizes that we used begin with 267 words, which was
the average document length in the DEFT challenge.
As features, we used log tf-idfweighted word types with words appearing
in fewer than five documents removed (though we experimented with a variety
of pre-processing methods, none of which had a profound effect on our results).
We used SVMs for binary classification and SVM-multiclass for multiclass classification. All of the results presented below are the averaged results of five-fold
cross-validation.
Binary classification: In performing binary classification, we were first faced
with the task of meaningfully splitting the groups involved into left-wing and
right-wing, a task that was further complicated by changes in groups and their
names over the ten-year study period and by inconsistencies in identification of
the groups in the data (e.g., Greens, Verts).14 From descriptions of the groups, we
classified as either broadly left or right 15 of the 18 affiliations observed in the
data,15 which we then grouped into the ten bins shown in Table 10.
Multiclass classification: We followed the example of the DEFT task in using
only the five largest groups for multiclass classification (see Table 10), excluding
the smaller right-wing groups. In multiclass classification, we found that tuning
the error cost C on a logarithmic range of values was especially important, and
that our best results were achieved with C on the order of 109.
13. If an MEP spoke significantly less than the document size, they were discarded from the
data. Even with the highest value for document size (6666 words), this depleted the data by only
2%. In our earlier Canadian experiments described above, we discarded small documents but
we did not subdivide large ones.
14. Group abbreviations usually appeared in French e.g., PSE rather than PES for the Party of
European Socialists even in the English data. Here, we use the predominant label.
15. Omitted were the non-inscrits (independents), the Technical Group of Independents (a
group described as politically heterogeneous), and the Alliance of Democrats and Liberals
(ALDE) (described variously as conservative liberals, or as centrist).
Table 10. European political groups as clustered, ordered from left-wing to right-wing.
For the purposes of binary classification, groups above the centrist group ALDE are
considered left-wing (L), and all groups below are considered right-wing (R). Asterisks
mark the five largest groups, which were used in the multiclass classification experiments.
Group
Speakers
in corpus
Description
L/R
*NGL
*PSE
*Greens
*ALDE
*PPE
*ECR
*EDD
*UEN
*EFD
*ITS
104
446
114
195
571
41
75
75
22
18
Communist / far-left
Social democrats
Green
Liberal / centrist
Conservative / Christian democrat
Conservative
Eurosceptic
National conservatism
Eurosceptic, national conservatism
Far-right nationalist
L
L
L
R
R
R
R
R
R
6.3
Results
Table 11 shows the accuracy of binary leftright classification with varying document sizes. The ten words most characteristic of each class are shown in Table 12.
Table 13 shows the accuracy of multiclass classification for varying document
sizes. The confusion matrix for multiclass classification is shown in Table 14; it
reflects a limited subset of the data, chosen so that each group was equally represented.16 Table 15 shows the ten words best characterizing each of the five classes.
Table 11. Precision, recall, and accuracy (%) of leftright classification on speech in the
European Parliament, with varying document sizes. Baseline accuracy (more frequent
class) is 5051%, varying slightly with document size.
Document size (words)
Precision
Recall
Accuracy
267
833
1667
3333
6666
62.6
67.6
69.9
72.9
77.6
65.2
70.1
71.9
77.5
81.3
62.3
67.4
69.8
73.9
78.5
16. Note that the confusion matrix reflects a sample of 1350 documents from the almost 3000
that we considered. We chose this sample so that each group had an approximately equal
number of documents. (Classification of the full set of documents tended to favour the groups
which were heavily represented, thus obscuring the measure of ideological similarity we were
looking for.)
Table 12. The top 10 English words characterizing each class in leftright classification
of speech in the European Parliament.
Rank
Left-wing
Right-wing
1
2
3
4
5
6
socialist(s)
unions
pse
employees
greens
scotland
subsidiarity
christian
strasbourg
competitiveness
healthy
prosperity
7
8
9
10
gender
equality
supports
myself
democrats
competitive
communist
truth
Table 13. Accuracy (%) of five-way multiclass classification of speech in the European
Parliament by political group, with varying document sizes. Baseline accuracy
(most frequent group) is 3839%, varying slightly with document size.
Document size (words)
Accuracy
267
833
1667
3333
6666
44.0
48.0
52.7
56.2
61.8
Table 14. Confusion matrix for multiclass classification of speech in the European
Parliament by political group. Column headings are our classifications, rows are
true affiliations. Boldface indicates correct classifications; italics indicates incorrect
classification of a group as an ideologically adjacent group. Shaded cells show confusion
between the PPE and the PSE.
NGL
PSE
Greens
ALDE
PPE
Total
NGL
PSE
Greens
ALDE
PPE
204
16
20
3
3
17
136
25
39
65
36
20
153
14
9
9
34
30
170
41
10
71
16
50
159
276
277
244
276
277
Total
Accuracy (%)
246
73.9
282
49.0
232
62.7
284
61.5
306
57.4
1350
61.8
111
Table 15. The top 10 English words characterizing each group in multiclass classification
of speech in the European Parliament.
Rank
NGL
PSE
Greens
ALDE
PPE
1
2
3
4
5
6
7
8
9
10
confederal
nordic
military
unemployment
profits
occupation
nato
liberalization
yugoslavia
militarisation
socialist
socialists
pse
institution
eplp
balanced
interinstitutional
millennium
repeatedly
portuguese
greens
alliance
nuclear
ngos
basque
scotland
comments
planes
ale
conflict
liberal
liberals
eldr
democrat
alde
alliance
rapporteurs
obvious
china
7
christian
subsidiarity
conservatives
morning
wrong
competitiveness
healthy
competitive
communist
phenomenon
6.4 Discussion
6.4.1 Comparison with DEFT results
With equal average document length, our multiclass accuracy was only marginally
better than the DEFT results of Forest et al. (2009) (about 5 percentage points over
baseline, rather than 2 points). However, as document size was raised to a maximum of 6666 words, accuracy increased steadily, up to 61.8%, about 23 points
over baseline. This suggests that the average DEFT size of 267 words is simply an
insufficient size for bag-of-words-based methods over such a noisy corpus.
6.4.2 Relative difficulty of classification tasks
The accuracy of multiclass compared to binary classification suggests that associating a speech with a specific group is not much harder than just classifying it as
left or right. This may, more than anything else, speak to the composition of the
European Parliament. Hix, Noury, and Roland (2007) suggest that, rather than
falling at some point on a line from left to right, European Parliament groups can
be placed in a space where the primary dimension is the traditional leftright axis
and the second dimension is a mixture of attitudes towards European integration
(in favour and against) and governmentopposition status in the EU (p. 217). In
addition, green sentiment, while often lumped in with liberalism, is not quite a
strict subset thereof (and implies a completely different vocabulary). These multiple dimensions complicate the task of binary ideological classification.
The confusion matrix for multiclass results (Table 14) may shed some light
on the relative ideological distances between groups. As we would wish, confusion is for the most part clustered around ideologically similar groups. Because
groups are arranged from left to right in order of ideology, this fact is reflected
by the tendency of confusion to cluster around the diagonal. The most surprising
result is the high amount of confusion between PSE (a socialist group) and PPE
(Christian democrats, the most conservative group we considered) (shaded cells).
This may be because these two groups had perhaps the least coherent feature
lists (see Section6.4.3 below). It may be significant that the two most accurately
classified groups, NGL and the Greens, also had the most subjectively coherent
feature lists.
6.4.3 Discriminative features
A few trends emerge from the lists of the top features for each group. The most
obvious is that MEPs tend to talk about their own groups. Hence, the top feature
for the Greens is greens, the top two features for the PSE are socialists and socialist,
and so on. This contrasts with Canadian MPs who we found (Section3.5 above)
tend to talk about their opponents more than themselves. This striking difference
demonstrates the domain-specificity of the features learned by the classifier. We
do however find some instances of MEPs talking about their opponents, most
notably the appearance of communist in the top 10 features of the PPE, the most
right-wing group we looked at. As might be expected, the contexts in which PPE
MEPs actually used the word were highly negative, phrases such as communist
tyranny. But clearly, whether MEPs talk about themselves or their opponents, the
names that each group tends to utter are important discriminators. Thus we see a
second reason why the results of Forest et al. (2009) were so poor; anonymization
of the groups removes crucial discriminators.
Some of the top feature lists are highly coherent with respect to the issues
of concern to the group. For example, among the top 50 features for the Greens
we find nuclear, organic, contaminated, ecological, toxic, culling, and depleted. The
top 50 features for NGL, the most left-wing of the groups, included wages, unemployment, capitalist, wage, inequality, and poverty. The top features for right-wing
PPE are less coherent, though as Diermeier et al. (2007) found of right-wing U.S.
senators, there tended to be a focus on cultural and moral issues: christian, moral,
conscience, faith, and euthanasia all appear in their top 100 features.
Some trends that the classifier seems to pick up on arent overtly ideological,
and indeed hint at the language of attack and defence. In the case of centre-left
group PSE, the classifier seems tuned to the language of felicitation, with words
like wholehearted, congratulations, congratulating, impressed, proud and achievement all in their top 50, whereas the centrist group ALDE seems to be associated
with censorious language: accountability, needless, shameful, shame, breaches.
7. Conclusion
Our results cast doubt on the generality of the results of research that uses words
as features in classifying the ideology of speech in legislative settings and possibly in political speech more generally. Rather, the language of attack and defence,
of government and opposition, seems to dominate and confound any sensitivity to
ideology. Such research therefore reduces in effect to the classification of support
or opposition, much as in the linguistic component of the work of Thomas et al.
(2006) described in Section1 above. However, even if our classifiers are construed
as distinguishing support from opposition, our results are much more accurate
than those of Thomas et al., even though we did not use any explicit component
for detecting agreement or disagreement between individual speakers. This may
be partly attributed to one of the differences between Canadian and U.S. politics:
Canadian parties have strong party discipline, and agreement between speakers
may be reliably inferred from shared party membership.
Our results contrast with the conclusions of Diermeier et al. (2007), who
argue from their own results that speakers words in debates in the U.S. Congress
are expressions or representations of an underlying belief system. Again, political differences might be a partial explanation of the difference. Perhaps the weak
party discipline of the U.S. and the separation of the Congress from the Executive
branch motivates greater attention to ideological substance in debates than does
the Canadian (Westminster-style) system in which an explicit governing party,
including the head of government and all cabinet ministers, is represented as
such in the legislature. This possibility is supported by our results from European
Parliamentary data. But this is speculation; our results have demonstrated a confound that must be taken into account in research on ideological classification of
speech in any context.
References
DEFT (2009). Actes de latelier de clture de la cinquime dition du Dfi Fouille de Textes,
Paris, 22 June 2009.
Diermeier, D., J.-F. Godbout, B. Yu and S. Kaufmann. 2007. Language and ideology in Congress.
Annual Meeting of the Midwest Political Science Association.
Forest, D., A. van Hoeydonck, D. Ltourneau and M. Blanger. 2009. Impacts de la variation
du nombre de traits discriminants sur la catgorisation des documents. Actes de latelier
de clture de la cinquime dition du Dfi Fouille de Textes, Paris, 22 June 2009, pp. 7588.
Germann, U. 2001. Aligned Hansards of the 36th Parliament of Canada. Available at www.isi.
edu/natural-language/download/hansard/
Greene, S. 2007. Spin: Lexical semantics, transitivity, and the identification of implicit sentiment.
PhD thesis, University of Maryland, College Park.
Grouin, C., B. Arnulphy, J.-B. Berthelin, S. El Ayari, A. Garcia-Fernandez, A. Grappy, M. HuraultPlantet, P. Paroubek, I. Robba and P. Zweigenbaum. 2009. Prsentation de ldition du DEfi
Fouille de Textes (DEFT09). Actes de latelier de clture de la cinquime dition du Dfi
Fouille de Textes, Paris, 22 June 2009, pp. 3550.
Hix, S., A.G. Noury and G. Roland. 2007. Democratic Politics in the European Parliament.
Cambridge: Cambridge University Press. DOI: 10.1017/CBO9780511491955
Jiang, M. and S. Argamon. 2008. Political leaning categorization by exploring subjectivities in
political blogs. Proceedings, 4th International Conference on Data Mining (DMIN 2008),
pp. 647653.
Lin, W., T. Wilson, J. Wiebe and A. Hauptmann. 2006. Which side are you on? Identifying perspectives at the document and sentence levels. Proceedings of the 10th Conference on Natural
Language Learning (CoNLL-X), pp. 109116.
Marx, M. and A. Schuth. 2010. DutchParl: The parliamentary documents in Dutch. Proceedings
of the 7th Conference on Language Resources and Evaluation (LREC 2010), Malta.
Mullen, T. and R. Malouf. 2006. A preliminary investigation into sentiment analysis of informal
political discourse. Proceedings of the AAAI Symposium on Computational Approaches to
Analysing Weblogs (AAAI-CAAW), pp. 159162.
Pennebaker, J.W. 2008. The meaning of words: Obama versus McCain. Weblog, 12 October
2008. Available at wordwatchers.wordpress.com/2008/10/12/
Pennebaker, J.W., C.K. Chung, M. Ireland, A. Gonzales and R.J. Booth. 2007a. The Development
and Psychometric Properties of LIWC2007. Available at www.liwc.net/LIWC2007 LanguageManual.pdf
Pennebaker, J.W., C.K. Chung, M. Ireland, A. Gonzales and R.J. Booth. 2007b. Linguistic
inquiry and word count (LIWC2007). Available at www.liwc.net
Riabinin, Y. 2009. Computational identification of ideology in text: A study of Canadian parliamentary debates. MSc paper, Department of Computer Science, University of Toronto,
January 2009. Available at www.cs.toronto.edu/compling/publications.html
Thomas, M., B. Pang and L. Lee. 2006. Get out the vote: Determining support or opposition
from congressional floor-debate transcripts. Proceedings of the Conference on Empirical
Methods in Natural Language Processing (EMNLP 2006), pp. 327335. DOI: 10.3115/
1610075.1610122
Yu, B., S. Kaufmann and D. Diermeier. 2008. Classifying party affiliation from political
speech. Journal of Information Technology in Politics 5(1), pp. 3348. DOI: 10.1080/
19331680802149608
chapter 6
Sentiment analysis
inparliamentaryproceedings
Steven Grijzenhout, Maarten Marx, and Valentin Jijkoun
University College London (UCL)/University of Amsterdam
This chapter addresses the question whether opinion-mining techniques can

successfully be used to automatically retrieve political viewpoints from parliamentary proceedings. Two specific preprocessing tasks were identified and
systematically evaluated: automatically determining subjectivity in the publications and automatically determining the semantic orientation of the subjective
parts. A corpus of recent parliamentary proceedings was collected and a gold
standard annotation was created on both subjectivity and orientation. Following
this, a number of models based on subjectivity lexicons and machine-learning
algorithms were evaluated. Machine-learning algorithms perform best, but
methods based on subjectivity lexicons also provide promising results. Based
on these results we can conclude that opinion-mining techniques applied to
political data score just as well as the state of the art in other more traditional
domains of opinion mining like product reviews and blogs.
1. Introduction
Opinion mining is a recent discipline concerned with automatically determining
the opinion a text expresses (Pang and Lee 2008). Opinion mining and sentiment analysis are terms that are used more or less synonymously in the literature,
and we will also use them interchangeably. In this paper, we evaluate whether
opinion-mining techniques can successfully be applied in political text analysis.
As data we use the verbatim transcriptions of the plenary meetings in the Dutch
House of Representatives. These documents are an important source of information on the position of political parties and individuals in the political arena. Our
research concerns the following central research question: Can opinion-mining
techniques be used to automatically retrieve political viewpoints from parliamentary proceedings? To answer this question (albeit indirectly), we evaluate whether
opinion-mining techniques are appropriate methods to analyze political data. This
approach is warranted as opinion-mining techniques have been tested elaborately
118 Steven Grijzenhout, Maarten Marx, and Valentin Jijkoun
on product reviews and blogs, but not yet on political data. We will show that the
data format of parliamentary proceedings is well suited to do sentiment analysis.
We then proceed to evaluate the two key steps in sentiment analysis: determining
subjective passages, and in these passages, determine the semantic orientation,
that is: do they express positive or negative attitude?
The paper is organized as follows: Section 2 provides background information on the various approaches to sentiment analysis; Section 3 describes the data;
Section 4 describes the task and the experiments we have done, and presents the
results; Section 5 reviews methods and results and compares them with other
research in this field.
2. Background
This research is part of a set of four research areas: opinion search, opinion mining, topic mining and recent research regarding Dutch parliamentary proceedings.
In this section these research areas are discussed briefly.
1. Opinion search. Opinion search is a relatively new branch of research. It aims
to enable users to search for opinions on any object (Liu 2007). Here, the entity
object is used to point to different concepts including products, persons, happenings or topics. Therefore opinion search can be helpful for a broad range of
applications, including review-related websites, blogs, business intelligence, government intelligence and politics. Most research covers opinion search applications in the context of blogs (web-logs) and review-related websites.
2. Opinion mining. Opinion mining concerns analyzing the opinion a text
expresses. Motivated by real-world applications researchers have considered a
wide range of problems in this area (Pang and Lee 2008). Esuli and Sebastiani
(2006) have organized these problems into three categories:
1. Determining subjectivity: the problem of determining whether a given text
passage has a factual nature or expresses an opinion.
2. Determining orientation (also called polarity): the problem of determining
whether a given subjective text expresses a positive or negative opinion.
3. Determining the strength of orientation: for example, weakly positive or
strongly negative.
A closely related task is extracting information on why the topic or product in the
text is considered positive or negative (Pang and Lee 2008). Other research problems include automatically determining the political color of a text, for example,
liberal or conservative (Mullen and Malouf 2006). The three categories identified
Chapter 6. Sentiment analysis inparliamentaryproceedings 119
by Esuli and Sebastiani (2006) account for the majority of the research in opinion mining. In this paper we evaluate algorithms for the first two categories:
automatically determining subjectivity, and determining the orientation of the
subjective passages.
3. Topical sentiment analysis. Here the goal is not only to determine that a passage expresses a certain attitude, but also to determine the object of the attitude.
Note that in many cases this is known from other sources than the passage (e.g., it
is listed separately in a product or movie review). A common approach is to apply
a categorization algorithm to a text and then perform a sentiment analysis (Osman
and Yearwood 2007).
4. Parliamentary proceedings. More and more large historical corpora of parliamentary proceedings become available for research. Examples include the British
Hansard and the Dutch Parlando. These are two search engines which provide the
digitized parliamentary proceedings in a machine-readable XML format. These corpora contain a wealth of information for historical political analysis but digital sustainability and good access to them remains a research challenge (Marx et al. 2010).
3. Data
We have tested our algorithms on the verbatim transcripts of the plenary meetings
of the Dutch House of Representatives (Tweede Kamer: Plenaire vergaderingen
(n.d.)). These are available in a variety of technical formats (PDF, Word, HTML,
XML). All of these, except the XML format, are meant for human reading and not
for machine processing. In this respect the Dutch data is typical for parliamentary
proceedings (Marx and Schuth 2010).
It is a technical challenge to transform some of these formats into useful
machine processible formats, especially for the older scanned and OCRed material (Marx and Schuth 2010). In order to determine political viewpoints from these
texts, the following information, easily detected by humans but not by machines, is
needed: for each word in the text we need to know whether it was spoken or not.
If so, by whom, the role or function of the speaker, when, and in which context.
For sentiment analysis, we also need a reliable segmentation of the text into words
and paragraphs.
These desiderata were best met when using the HTML version of the proceedings data. These are available from the Dutch parliament directly, one day after
each meeting, as a draft version (from www.tweedekamer.nl). The transcripts were
downloaded and automatically transformed into the XML format described in
Marx and Schuth (2010). An example is provided in the Appendix.
4. Assessing subjectivity and orientation

In this section, we present several sentiment analysis techniques, apply them to
our dataset, and systematically evaluate their quality using a manually annotated
corpus. We perform the first two classification tasks according to the schema of
Esuli and Sebastiani (2006). First we determine whether a text is subjective or
objective. Second, for the subjective texts, we determine whether they have a positive or negative orientation. Before we start with the classifiers we first must decide
on the level of detail on which classification is done and on how to create the gold
standard corpus.
4.1
Classification level
Before classification can commence, the level at which it will be conducted needs
to be chosen. Different levels are used in the literature:
Document level (Yu and Hatzivassiloglou 2003): whole documents are labeled.
For example, a document can have an overall orientation that is classified as
positive.
Block level (Osman and Yearwood 2007): the text is cut into several blocks and
each block is labeled independently. This is most often used in unstructured
data like blog pages.
Paragraph level (Kamps and Marx 2001): each paragraph is labeled.
Sentence level (Riloff and Wiebe 2003; Wilson et al. 2003; Furuse et al. 2007:
each sentence is labeled.
Word level (Yu and Hatzivassiloglou 2003; Kim and Hovy 2005; McKeown and
Hatzivassiloglou 1997): individual words are labeled.
Classification at document level and word level is unsuitable for identifying political viewpoints in parliamentary proceedings. Document level classification means
a whole meeting is treated as an individual entity, and marking it will give no
particular views of individual parties or political persons. It is too general to be
of value. In contrast, classification at the word level is too detailed, and will not
contain enough contextual information to connect sentiment to a particular viewpoint or topic.
Classification at the sentence level also has problems with contextual information since individual sentences will contain references to adjacent sentences and
topics. For example, the sentence That is okay. contains an opinion, but we do
not know what the opinion is about. Arguments containing a viewpoint are often
expressed in multiple sentences. This leaves us with the choice between either
block level or paragraph level classification. Since a paragraph is considered a

natural block, this has been considered most appropriate. The source data was
already split into paragraphs (using the p-element in HTML), so no further processing was needed.
4.2
Gold standard corpus
Evaluation of opinion retrieval algorithms mostly relies on a comparison with

human annotations from the same corpus (Ku et al. 2006.; Osman and Yearwood
2007). To evaluate the performance of the algorithms on the Dutch parliamentary proceedings, a gold standard was developed. We aimed at annotating around a thousand paragraphs. For efficiency reasons, the paragraphs were
extracted from as few documents as possible. We randomly chose two meetings
(March 5th and April 21st, 2009) which contained enough (1201) paragraphs.
Paragraphs spoken by the chairman were not annotated because the chairman
does not take part in the discussions on political issues, but instead tries to keep
the meetings on track.
The first task was to annotate whether a paragraph contains an opinion or not.
If there is an opinion present, the paragraph is considered subjective. Otherwise
the paragraph is considered objective. Examples of objective and subjective paragraphs are given in the Appendix. Two human annotators were used, both with
Dutch as their mother tongue. The paragraphs were printed and split evenly
between them. A face-to-face explanation of the intention of the research and
their task of annotating the paragraphs was provided. The annotators marked each
paragraph as subjective or objective. This was judged by reviewing each individual
paragraph against a definition of subjectivity. The definition of subjectivity that
was used is based on the literature (Banea et al. 1999; Kim and Hovy 2004; Riloff
and Wiebe 2003; Wiebe and Riloff 2005; Wiebe et al. 1999), the Compact Oxford
English Dictionary definition of opinion, and the Dutch Van Dale online dictionarys definition of mening)(Van Dale online dictionary). Our definition is:
If the primary intention of a piece of text is an objective presentation of material
that is factual to the reporter, and does not contain a judgment or emotion, the
text is objective. Otherwise the text is subjective.
The second task was to annotate the semantic orientation of each subjective paragraph. As mentioned, the orientation of a text is whether it expresses a positive or
negative opinion. The same two annotators were used. This time, however, instead
of splitting the paragraphs evenly between them, the two annotators individually
marked all of the subjective paragraphs. Discontinuities between the annotators
were afterwards resolved via mutual consultation. To have two judgments meant
that the inter-annotator agreement could be monitored (see below).
In the literature, a clear definition of positive and negative orientation is hard
to find. Most of the time multiple human annotators are used to judge a corpus based on their intuition or common sense (Furuse et al. 2007; Jijkoun and
Hofmann 2009). Based on research by Osgood et al. (1957) the semantic orientation on which we wish to classify the paragraphs is the evaluative factor: good/
bad. They proved that this factor is the most significant influence on variation in
data. A definition of orientation based on this research can be found in Turney
(2001): a phrase has a positive semantic orientation when it has good associations and a negative semantic orientation when it has bad associations. Turney
and Littman (2003) also distinguish between positive evaluation (e.g., praise) and
negative evaluation (e.g., criticism) respectively. From these sources, the following definition to classify this binary orientation was formulated, leaning heavily
on Osgood et al. (1957). The annotators were instructed to use this definition in
their task:
A text has a positive orientation when it has good associations, or contains a positive evaluation (e.g., praise). The text has a negative orientation when it has bad
associations or contains negative evaluations (e.g., criticism).
Corpus statistics. In the transcripts of the meetings of March 5th and April 21st,
2009, a total of 1201 paragraphs were annotated, of which 590 (49.1%) were
annotated as subjective. Out of these 590 subjective paragraphs, 251 (42.5%) were
annotated as positive, and 339 (57.5%) as negative. Because two annotators were
used, inter-annotator agreement could be calculated. The overall agreement is
71.4%. There was hardly any difference in agreement between the positive and the
negative paragraphs (175/251=69.5% and 246/339=72.4%, respectively). Cohens
is 0.423.
Conclusions. Because of the use of definitions, the annotation tasks were easy to
explain to the annotators. Also, because strict definitions were used, the annotators did not need to have specific domain knowledge. According to some, an
analytic definition of opinion is impossible (Kim and Hovy 2004). Still, even with
the strict instructions, inter-annotator agreement was low: overall agreement on
semantic orientation between the two annotators was 71.4% and = 0.423. These
rather low values are however common for sentiment analysis tasks: e.g., (Kim
and Hovy 2005) classified 174 sentences by three annotators and found a pair-wise
agreement of 73% and a kappa value of 0.49.
4.3
Automatically determining subjectivity
We now describe and evaluate a number of algorithms for determining subjectivity of texts. Technically, these algorithms are binary classifiers: for each input
they decide whether it is subjective or not. Algorithms for such a task fall into
two categories: based on handcrafted rules, or (machine) learned from examples.
Algorithms in the first category make use of so called subjectivity lexicons. We will
evaluate two algorithms based on lexicons and three based on machine learning.
Algorithms based on subjectivity lexicons

In this approach, the focus is on the number of occurrences of each term. The
exact ordering of the terms in a text is not important (Manning et al. 2008). Most
often the individual words are given a certain subjectivity score based on a set of
opinion words (the subjectivity lexicon) (Ding and Liu 2007). The models then
present a way to calculate the subjectivity of the whole text based on the individual
collection and frequency of these words. We discuss two models from Kim and
Hovy (2005) which implement this idea.
Model 1 counts the total valence score of all words in the paragraph. The basis
of this model is that paragraphs dominated by words considered to be subjective
tend to be opinion bearing. Individual words in the paragraph are extracted and
given a score of 0, 1 or 2, in which a score of 2 is considered to be very subjective
and a score of 0 not subjective. A Dutch sentiment wordlist developed by Jijkoun
and Hofmann (2009) was used to rate the words. Words not present in the wordlist are considered to be not subjective and have been given a score of 0. A cut-off
threshold had to be selected in order to determine when a paragraph is judged to
be subjective or objective. Experimentation has been conducted with threshold
values between 0 and 20.
Model 2 checks the presence of a single strong valence word. The assumption
underlying this model is that the presence of one strong valence word is enough to
indicate subjectivity. The Jijkoun and Hofmann sentiment lexicon (2009) is used
and a cut-off threshold is set to determine at which score a paragraph is considered
to be subjective. Because the wordlist by Jijkoun and Hofmann (2009) contains
scores of 0, 1 and 2, where 0 indicates neutrality, the performance of the algorithm
is evaluated on cut-off thresholds of 1 and 2.
Machine-learning algorithms
Machine-learning algorithms differ from models based on subjectivity lexicons
in that they automatically train themselves to classify the data. The methods we
use are called supervised methods because a labeled data set is needed to train
the classifier. For this we use our gold standard. The following machine-learning
algorithms are used (Manning et al. 2008):
NaiveBayes;
IBk nearest-neighbour, with k=1;
Support Vector Machine (SVM) SMO;
ZeroR (as the baseline).
The toolkit Weka 3.6.1 is used to train and evaluate the machine-learning classifiers
(WekaWiki: Primer (n.d.)).
Results
The performances of the algorithms are based on accuracy, precision, recall and
F-measure (Manning et al. 2008). The machine learning algorithms are evaluated
using tenfold cross-validation.
As described above, Model 1 uses a cut-off threshold above which the text is
classified as subjective. The performance of the model at different cut-off thresholds can be found in Table 1. The highest scores are in bold font. Because the aim
of the subjectivity classification is to retrieve paragraphs that are subjective, we
are interested mostly in the results of the TRUE-class. (The TRUE class consists
of the set of subjective paragraphs.) A higher threshold will lower recall, and will
increase precision. This is as expected, as a higher cut-off threshold will include
less subjective markers (Kim and Hovy 2005). The table should be interpreted as
follows: A threshold of >n means that a paragraph is classified as subjective if the
sum of the valence scores in the paragraph is larger than n. Thus the threshold
>0 means that a paragraph is subjective if it contains at least one word with
subjectivity score 1. We see that with threshold >1, we find 93% of all subjective
paragraphs (recall=.929) and that 55% of those classified as subjective are indeed
subjective (precision=0.550).
Model 2, based on Kim and Hovy (2005), also uses a cut-off threshold. Here the
thresholds have a different meaning: threshold 1 means that the paragraph contains
at least one subjective word (thus with score 1 or 2); threshold 2 that it contains at
least one word with score 2 (a highly subjective word). The results are in Table 2.
Threshold 1 has the best tradeoff between precision and recall (F=.677). Note that
Model 2 with threshold 1 is exactly the same as model 1 with threshold>0: both
classify a paragraph as subjective if it contains at least one subjective word.
The best results of the two models and the results of the machine-learning
algorithms are shown in Table 3. In reaching these results, experiments were conducted for smoothing them out, for example, all words in the paragraphs were
converted to lowercase. These experiments did not improve results significantly.
In fact, converting the paragraphs to lowercase even deteriorated results.
Table 1. Results of cut-off threshold values using model 1 based on Kim and Hovy (2005).
Threshold
Baseline (all subjective)
>0
>1
>2
>3
>4
>5
>6
>7
>8
>9
> 10
> 11
> 12
> 13
> 20
Results on TRUE class

Precision
Recall
F-measure
0.491
0.521
0.550
0.570
0.591
0.605
0.636
0.653
0.671
0.671
0.684
0.689
0.695
0.688
0.705
0.793
1.0
0.966
0.929
0.854
0.775
0.686
0.615
0.546
0.478
0.395
0.337
0.275
0.232
0.186
0.146
0.039
0.659
0.677
0.691
0.684
0.671
0.643
0.625
0.595
0.558
0.497
0.452
0.393
0.348
0.293
0.242
0.074
Table 2. Results of cut-off threshold values using model 2 based on Kim and Hovy (2005).
Threshold
1
2

Precision
Recall
F-measure
0.521
0.596
0.966
0.666
0.677
0.628
Table 3. Results of all approaches on classifying subjectivity (using optimal threshold

results at K&H models for weighted results and TRUE class results).
Model
K&H model 1 (threshold >1)
K&H model 2 (threshold 1)
NaiveBayes
Ibk
SMO

Precision
Recall
F-measure
0.550
0.521
0.607
0.563
0.638
0.929
0.966
0.802
0.593
0.610
0.691
0.677
0.691
0.578
0.624
Conclusions
The performance of Model 1 on the TRUE class is amongst the highest with an
F-measure of 0.691: it finds almost all subjective paragraphs and classifies just over
half of them correctly. How good are these results? For that we compare our scores
to those obtained by Kim and Hovy (2005). However they report F measures for
the weighted results of both classes.
The F-measure of Model 1 on our data is at its peak at 0.545 (threshold >6). Our
implementation of Model 1 performs better on our data than the original model
performed on TREC 2003 data that only achieved a F-measure of 0.425. The implementation of Model 2 also performed better on our data than the original model
on TREC 2003 data, achieving an F-measure of 0.534 (threshold 2) as opposed to
0.514. Thus we can conclude that the results of the methods based on subjectivity
lexicons are very promising, as they perform relatively well on political data.
From the machine-learning algorithms, NaiveBayes performs best overall
with a weighted F-measure of 0.640 and an F-measure on the TRUE class of 0.691.
If we select ZeroR, which predicts a class based on the mode (thus in our case:
it classifies all paragraphs as objective), as baseline, NaiveBayes performs significantly better than ZeroR (one tailed test, confidence level 0.99). The SVM algorithm
SMO also produces decent results significantly better than ZeroR (one tailed test,
confidence level 0.99). The weighted F-measure of 0.638 comes in range of the
NaiveBayes weighted F-measure of 0.640. On classifying the TRUE class, however,
NaiveBayes would still be the preferred algorithm of choice with an F-measure of
0.691 as opposed to SMOs F-measure of 0.624.
4.4 Automatically determining semantic orientation
We now evaluate algorithms which automatically determine semantic orientation. Like the algorithms determining subjectivity, these algorithms are classifiers. In line with the literature, we built binary classifiers which classify subjective
paragraphs as positive or negative. Again there are lexicon based approaches and
machine learning algorithms.
Algorithms based on subjectivity lexicons

The algorithm is based on the model by Edens et al. (2006) and uses the wordlist
by Jijkoun and Hofmann (2009). The algortithm classifies all words as positive,
negative or neutral. The scores +2 and +1 are considered positive, and 1 or 2
are considered negative. The algorithm also takes into account that two adjacent
polar words of the same orientation influence each other. A factor is calculated
based on the distance between the two polar words, with a maximum distance of
10. The score of the original polar word is then multiplied by this factor. The following equation is used to calculate the new wordscore:
wordscore = wordscore (1 + 10/distance )
10

After all the wordscores in the paragraph have been calculated, they are added up.
If the final score is above 0, the paragraph is classified as positive, otherwise the
paragraph is negative.
The model based on Chesley et al. (2006) combines a subjectivity lexicon and
machine learning. The expectation of this model is that the distribution of positive
and negative adjectives, and positive and negative verb classes, shows regularities.
Furthermore, it assumes that the orientation of adjectives can be described by the
majority orientation class of their synonyms. We have implemented this model
as follows:
First, a part of speech (POS) tagger (TreeTagger 3.2) is used to identify all
verbs and adjectives. Next, for all adjectives, the synonyms are scraped from the
website www.synonyms.net. In the original implementation by Chesley et al.
(2006), Wikipedias dictionary is used because of its coarse-grained content. We
used www.synonyms.net instead because the Dutch version of Wiktionary is not
sufficiently developed yet. All collected synonyms are matched against a wordlist
of positive and negative adjectives. The wordlist is created by merging the adjectives of Jijkoun and Hofmann (2009) and the negative and positive adjectives collected by Kamps and Marx (2001). The majority class of the synonyms has been
assigned to the adjective. The verbs are assigned to a positive or negative class
based on the lexicon by Jijkoun and Hofmann (2009). As output, the model provides a list of information on each paragraph consisting of:
1.
2.
3.
4.
Number of positive adjectives

Number of negative adjectives
Number of positive verbs
Number of negative verbs
To this list, the gold standard classification on orientation belonging to the paragraph is added. Finally, using the information gathered on the paragraphs, Chesley
et al. (2006) used a SVM algorithm to classify the paragraphs. They opt for the use
of a SVM algorithm because they believe it to be robust for sentiment classification and handling noisy data (Mishne 2005). We use Wekas SVM algorithm SMO,
but experiment with NaiveBayes, IB1, and ZeroR as well (see Table 4). NaiveBayes
gave the best results.
Table 4. Results of classification on orientation using the output of the model based on
Chesley et al. (2006).
Classifier
Precision
Recall
F-measure
SMO
NaiveBayes
IB1
ZeroR
0.567
0.597
0.553
0.330
0.581
0.602
0.558
0.575
0.560
0.599
0.554
0.419
Machine-learning algorithms
Again, four machine-learning algorithms are selected to represent this category.
NaiveBayes
IB1 nearest-neighbour
Support Vector Machine (SVM) SMO
ZeroR
Again, the Weka toolkit is used to evaluate these algorithms.
Results
The machine-learning algorithms are evaluated using ten fold cross-validation.
Similarly to the algorithms determining subjectivity, all algorithms are evaluated
on precision, recall and F-measure. Because the four features used by the model
based on Chesley et al. (2006) are numeric, discretization of the data could be
conducted to improve results. Experiments have been conducted with different
bin sizes for each classifier. The optimal results can be found in Table 4. The following parameters were used:
SMO: Discretization with Wekas option findNumbins. This option lets Weka
choose an appropriate amount of bins.
IB1: no discretization.
NaiveBayes: no discretization.
NaiveBayes produces the best results on all measures: almost 60% of all classifications is correct at a recall value of .602. It is difficult to compare these results to
the results found by Chesley et al. (2006) since they classify on document level
of a blog post instead of on paragraph level. The results nevertheless, allow us to
conclude that the SVM classifier is performing well on the data. The results support the claim of Chesley et al. (2006) and Mishne (2005) that SVM is a robust
classifier for sentiment classification.
A comparison of all algorithms used to classify semantic orientation can be
found in Table 5. Some smoothing experiments were conducted, but again did
not improve results. In contrast to subjectivity results, the SVM machine learning
classifier scores best on all fronts regarding orientation classification. It is followed

by NaiveBayes. Both perform significantly better than the other models used (one
tailed test, confidence level 0.99). The combination of collecting paragraph statistics
and using a machine-learning algorithm gives promising results with an F-measure
of 0.599. If more characteristics are collected, performance may increase.
Table 5. Results of classifications by semantic orientation.
Model
Precision
Recall
F-measure
model based on Edens et al. (2006)

model based on Chesley et al. (2006) with NaiveBayes
Ibk
NaiveBayes
SMO (SVM)
ZeroR
0.369
0.597
0.601
0.652
0.677
0.330
0.517
0.602
0.561
0.651
0.676
0.575
0.419
0.599
0.556
0.652
0.677
0.419
5. Conclusions
The aim of this chapter was to evaluate the appropriateness of sentiment analysis
techniques which are developed for blogs and product reviews, for the analysis of
political texts. We first summarize our technical results and then return to their
impact on the main research question.
We studied six algorithms to automatically detect opinion-bearing paragraphs. Machine-learning and lexicon-based algorithms scored about equally
well. NaiveBayes performed best with a weighted F-measure of 0.640 and an
F-measure on the TRUE class of 0.691. Model 1, based on subjectivity lexicons,
achieved exactly the same F-measure. Next, six algorithms were studied which
automatically detect the orientation of the subjective paragraphs. The algorithms
classified paragraphs as either positive or negative. Machine-learning algorithms
again dominated the results. NaiveBayes reached an F-measure of 0.652, but the
SVM implementation in Weka called SMO performed best with an F-measure of
0.677. Both performed significantly better than the other algorithms. These results
support the claim that SVM provide a solid method for sentiment classification
(Chesley et al. 2006; Mishne 2005). It can furthermore be concluded that a model
collecting paragraph characteristics and then classifying the paragraphs using
machine-learning algorithms provides promising results.
Considering the performances of the classification algorithms, we conclude
that results are approximately in line with results found in the literature. An
F-measure approaching 0.7 is a common achievement, and can therefore be considered to be a respectable result. In other words, opinion-mining techniques are
suitable to automatically retrieve subjective paragraphs from texts and to annotate

their orientation. This shows that todays opinion-mining techniques can be successfully applied to Dutch, political, semi-structured transcripts.
With both techniques scoring about equally well, we advise on using either lexicon based or machine-learning for political texts based on other grounds. It seems
that a machine learning approach might be preferable. First, using Crowdsourcing
platforms like Amazon Mechanical Turk, it becomes easy, fast, and inexpensive to
create a large number of training examples. Even though the task is difficult and
we cannot expect high inter-annotator agreement, this option can work fine. In
usual experiments; each task is performed by five annotators. Only examples with
a strong majority agreement (e.g., 4 out of 5 agree) could be used. Second, politicians are creative language users, and political language as a result changes fast. It
is therefore advisable to train a learning algorithm on examples, rather than trying
to capture the political language in a lexicon, especially since orientation in political data is topic-dependent. One note of caution is called for though. In case of a
lack of resources, as in this study, the machine-learning algorithms are trained and
tested on the same set. This might have the effect of overtraining the algorithms.
How can these results be used to retrieve political viewpoints or party positions? For retrieving a viewpoint we also need to know about which topic an
opinion was given. This can be done by combining a topical paragraph classifier
with the here developed sentiment classifier. Then there are still several ways to
distill a party position on a topic based on a collection of paragraphs about that
topic spoken by many members of that party. Thus a separate evaluation seems
needed to answer the question.
The chapter by Hirst et al. (this volume) finds that lexical methods (as we
use here) do not retrieve party positions but rather government versus opposition positions. Whether this also holds for opinionated paragraphs is a matter
for further research. Here we can only give a first impression. We have analyzed
our two days of annotated debates with respect to this question and obtained
mixed results1 (see Table 6). The debates of April 21, 2009 show an almost perfect
1. The editors of this volume made the following notes on these two days. On Tuesday April 21,
the results show a perfect division between government and parliament. On Tuesdays the Dutch
parliament has Question Time. This might explain the division between government and the
government-parties on the one hand and parliament, more specifically the opposition parties,
on the other hand. The opposition parties attack the government and the government praises
itself and is praised by the government parties. March 5th, on the other hand, is a Thursday with
more elaborate and detailed debates in which parties (apparently) take less clear positions. This
division between question time and debates is also noted by Hirst et al. (this volume). They also
find a stronger division between government and parliament during Oral Question Period than
for Debates.
dichotomy between speakers from the government and parties in the government (ChristenUnie, CDA and PvdA), and the other parties. The only exception
is minister Verdonk, with just 6 paragraphs. All opposition parties have more
negative than positive paragraphs, while it is exactly the other way around in the
other group. The other day does not have this clear division between government
and opposition. On this rather gloomy day (on which even the government has
almost twice as many negative praragaphs than positive), only the Christian parties (CDA, ChristenUnie and SGP) and the Greens (GroenLinks) are more positive in general.
Table 6. Results of classifications by semantic orientation.
Thursday March 5th, 2009
Party
ChristenUnie
CDA
GroenLinks
SGP
PvdA
D66
Government
VVD
PVV
SP
Verdonk
PvdD
Tuesday April 21st, 2009
POS/NEG POS
1.3
1.1
1.0
1.0
0.7
0.7
0.6
0.5
0.4
0.2
0.0
0.0
4
27
14
6
15
10
33
16
4
10
0
0
NEG
Party
3
24
14
6
21
15
59
34
11
41
0
0
ChristenUnie
Verdonk
CDA
PvdA
Government
SGP
PVV
GroenLinks
VVD
SP
D66
PvdD
POS/NEG POS
3.7
3.0
1.8
1.6
1.2
1.0
0.9
0.8
0.8
0.3
0.0
0.0
11
3
25
11
20
5
12
7
10
8
0
0
NEG
3
1
14
7
17
5
13
9
13
28
0
1
EXPLANATION: We count the number of positive and negative paragraphs spoken by members of that
party for each day, for each party. The second attribute is the ratio between the numbers of positive and
negative paragraphs. If the ratio is higher than 1, the party is overall positive that day, below 1 indicates a
more gloomy day.
Another factor that heavily influences the findings is the style of a debate. In the
Dutch House of Commons, interruptions of speeches are made frequently and
are recorded in the proceedings together with answers to interruptions. This may
cause that governing parties also use more negative terms (e.g., in their answers
to negatively phrased interruptions). An interesting dataset to test the findings of
Hirst et al. (this volume) are the Danish parliamentary proceedings. Denmark has
a minority cabinet since the early nineties. It could be interesting to see how the
dynamic between government and parliament in this situation is reflected in the
language of attack and defense.
References
Banea, C., R. Mihalcea, and J. Wiebe. 2008. A bootstrapping method for building subjectivity
lexicons for languages with scarce resources. LREC 2008.
Chesley, P., B. Vincent, L. Xu, and R. Srihari. 2006. Using Verbs and Adjectives to Automatically
Classify Blog Sentiment. AAAI Spring Symposium Technical Report SS-06-03.
Compact Oxford English Dictionary: opinion. (n.d.). (O. U. Press, Producer) Retrieved 06 05,
2009 from Compact Oxford English Dictionary: http://www.askoxford.com:80/concise_
oed/opinion
Ding, X., and B. Liu. 2007. The utility of linguistic rules in opinion mining. Proceedings of the
30th Annual international ACM SIGIR Conference on Research and Development in information Retrieval, pp. 811812. Amsterdam: ACM, New York, NY.
Edens, J., M. Liem., T. Mensink, R. Weve, and L. van Zande. 2006. Measuring Politics. University
of Amsterdam, Amsterdam.
Esuli, A. and F. Sebastiani. 2006. Determining term subjectivity and term orientation for opinion mining. Proceedings of the Eleventh Conference on European Chapter of the Association
for Computational Linguistics. Trento, Italy: European Chapter Meeting of the ACL. Association for Computational Linguistics., pp. 193200.
Furuse, O., N. Hiroshima, S. Yamada, and R. Kataoka. 2007. Opinion sentence search engine
on open-domain blog. Proceedings of the 20th International Joint Conference of Artificial
Intelligence (IJCAI2007).
Jijkoun, V. and K. Hofmann. 2009. Generating a non-English subjectivity lexicon: Relations that
matter. Second Conference of the European Chapter of the Association for Computational
Linguistics (EACL-09).
Kamps, J. and M. Marx. 2001. Words with attitude. 1st International WordNet Conference,
pp.332341.
Kim, S.-M. and E. Hovy. 2004. Determining the sentiment of opinions. Proceedings of
COLING-04, pp. 13671373. Geneva, Switzerland.
Kim, S.-M. and E.H. Hovy. 2005. Automatic detection of opinion bearing words and sentences.
Second International Joint Conference on Natural Language Processing.
Ku, L., Y. Liang, and H. Chen. 2006 Tagging heterogeneous evaluation corpora for opinionated
tasks. LREC 2006.
Liu, B. 2007. Web Data Mining: Exploring Hyperlinks, Contents and Usage Data. Heidelberg:
Springer.
Manning, C., P. Raghavan, and H. Schtze. 2008. Introduction to Information Retrieval.
Marx, M., N. Aders and A. Schuth. 2010. Digital sustainable publication of legacy parliamentery
proceedings. In Proceedings dg.o 2010.
Marx, M. and A. Schuth. 2010. DutchParl. A corpus of parliamentary proceedings in Dutch. In
Proceedings LREC 2010. pp. 36703677.
McKeown, K. and V. Hatzivassiloglou. 1997. Predicting the semantic orientation of adjectives.
Proceedings of the 35th annual meeting of ACL.
Mishne, G. 2005. Experiments with mood classification in blog posts. 1st Workshop on Stylistic
Analysis Of Text For Information Access.
Mullen, T. and R. Malouf. 2006. A preliminary investigation into sentiment analysis of informal
political discourse. AAAI Symposium on Computational Approaches to Analysing Weblogs
(AAAI-CAAW), pp. 159162.
Osgood, C., G. Suci, and P. Tannenbaum. 1957. The measurement of meaning. Urbana, IL:
University of Illinois Press.
Osman, D. and J. Yearwood. 2007. Opinion search in web logs. Proceedings of the Eighteenth
Conference on Australasian Database, 63. Ballarat, Victoria, Australia.
Pang, B. and L. Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in
Information Retrieval, 2 (1-2), pp. 1135.
Riloff, E. and J. Wiebe. 2003. Learning extraction patterns for subjective expressions. Conference
on Empirical Methods in Natural Language Processing (EMNLP-03), pp. 105112.
TreeTagger 3.2. (n.d.). From TreeTagger 3.2: http://www.ims.uni-stuttgart.de/projekte/corplex/
TreeTagger/
Turney, P. 2001. Thumbs up or thumbs down?: Semantic orientation applied to unsupervised
classification of reviews. Proceedings of the 40th Annual Meeting on Association For Computational Linguistics pp. 417424. Philadelphia, Pennsylvania: Annual Meeting of the ACL.
Association for Computational Linguistics, Morristown, NJ.
Turney, P. and M. Littman. 2003. Measuring praise and criticism: Inference of semantic orientation from association. ACM Trans. Inf. Syst., 21(4), pp. 315346. DOI:
10.1145/944012.944013
Tweede Kamer: Plenaire vergaderingen. (n.d.). Retrieved 06 03, 2009 from Tweede Kamer der
Staten Generaal: http://www.tweedekamer.nl/vergaderingen/plenaire_vergaderingen/
index.jsp
Van Dale online dictionary: mening. (n.d.). (V. Dale, Producer) Retrieved 06 05, 2009 from
Mening: http://www.vandale.nl/vandale/opzoeken/woordenboek/?zoekwoord=mening
WekaWiki: Primer. (n.d.). Retrieved 06 10, 2009 from Weka-Machine Learning Software in Java:
http://weka.wiki.sourceforge.net/Primer
Wiebe, J.M., R.F. Bruce, and T.P. OHara. 1999. Development and use of a gold-standard data
set for subjectivity classifications. Proceedings of the 37th annual meeting of the Association
for Computational Linguistics on Computational Linguistics, pp. 246253. College Park,
Maryland: Association for Computational Linguistics. DOI: 10.3115/1034678.1034721
Wiebe, J. and E. Riloff. 2005. Creating subjective and objective sentence classifiers from unannotated texts. In Computational Linguistics and Intelligent Text Processing Vol. 3406/2005,
pp. 486497. Heidelberg: Springer.DOI: 10.1007/978-3-540-30586-6_53
Wilson, T., D. Pierce., and J. Wiebe. 2003. Identifying opinionated sentences. Proceedings of the
2003 Conference of the North American Chapter of the Association For Computational Linguistics on Human Language Technology:Demonstrations. 4, pp. 3334. Edmonton, Canada:
North American Chapter of the Association for Computational Linguistics.
Yu, H. and V. Hatzivassiloglou. 2003. Towards answering opinion questions: Separating facts
from opinions and identifying the polarity of opinion sentences. Proceedings of the 2003
Conference on Empirical Methods in Natural Language Processing, pp. 129136. DOI:
10.3115/1119355.1119372
Appendix
Figure 1. Example of the data format in XML annotated with the gold standard.
chapter 7
The qualitative analysis of political documents

Jared J. Wesley
University of Alberta, Department of Political Science
Qualitative document analysis remains one of the most common, yet methodologically misunderstood, components of political science research. While
analysts are accustomed to incorporating manifestos, speeches, and other
documents as evidence in their studies, few approach the task with the same
level of understanding and sophistication as when applying quantitative
methods. Building bridges between the two traditions, this chapter suggests
guidelines for the rigorous, qualitative study of political documents. The
discussion includes a novel examination of materials from the Poltext Project
collection a compilation of documents from across the Canadian provinces.
The paper concludes that, whether approaching their work from a quantitative
or non-quantitative perspective, researchers must adhere to similar disciplinary standards if their findings are to be considered trustworthy contributions
to political science.
Introduction
Seldom do textbooks or courses in political science methodology devote any
attention to manual qualitative document analysis (qda). Dominated by the
discussion of interviews, focus groups, (quantitative) content analysis, experimentation, and field studies, few offer any treatment of qda, whatsoever (but
see Wesley 2011a, 2011b, 2011c). When it is discussed, the qualitative analysis
of political texts is typically included under broad topics like unobtrusive or
archival research, or conflated with the coding of transcripts or field notes
(George 2006:135; Platt 2006a:83). This lack of detailed methodological discussion is disconcerting, considering that most political scientists have, at
some point, incorporated elements of textual interpretation as evidence in their
research. To move beyond armchair interpretation, however, such analyses
must be guided by the same level of rigour as those in the quantitative fields
136 Jared J. Wesley
(Wesley 2011b). The following paper suggests a series of such guidelines, based
on a comparison with traditional modes of quantitative content analysis, a
review of recent qda studies, an examination of similar approaches in other
social science disciplines, and a novel study of documents drawn from the
Poltext Project collection.
Three types of Qualitative Document Analysis (QDA)

A review of the literature reveals that political scientists tend to employ three
main forms of qualitative inquiry when it comes to the study of documents. Each
involves its practitioners in a search for a different set of themes, using conventional, directed, or summative tools.
The first and most common qualitative approach to political documents
involves rhetorical analysis. Rhetoric is the art of persuasion; thus students of
rhetoric in political documents must investigate what the author of a text is trying
to achieve and the strategies employed to that end. In this vein, documents may
be analyzed according to their style of writing (e.g., formal or informal), type of
diction (e.g., jargonistic or folksy), tone (e.g., moderate or extreme), use of certain
devices (e.g., metaphors or sarcasm), reference to keywords or formulaic phrases
(e.g., patriotism or feminism), invocation of certain emotions (e.g., certainty or
fear), use of icons or figures (e.g., photographs or charts), allusion to imagery or
symbolism, or any other element of the authors delivery.
In this sense, the manifest content of the appeal is of less concern to rhetorical analysts than purpose for, and the means by, which the message is conveyed.
In their effort to provide A Rhetorical Perspective on the 1997 British Party
Manifestos, for example, Craig Allen Smith and Kathy B. Smith (2000:458) examined how political parties used the resources of language to negotiate a shared
understanding among members of the electorate. As they describe, Any election campaign is a contest between a we and one or more thems in which each
party attempts to maximize the we relative to the thems Political parties try
to accomplish this through the management of themes, visions, symbols, needs,
preferences, and reasons.
To study these rhetorical strategies, Smith and Smith examined each major
partys campaign literature, comparing their ideological content, style of writing,
iconic layout, and aesthetic packaging. The Conservative Manifesto is a startlingly impersonal document, they reported.
Chapter 7. The qualitative analysis of political documents 137
It shows only two human faces, virtually identical pictures of a suited [Prime
Minister] John Major (one indoors and smiling, the other outdoors with windtousled hair and a half-smile. All other Britons are silhouetted, shadowed, or
masked from view by looking into a microscope, facing a computer screen, wearing surgical masks, or talking on cell phones. These seem not be persons at all but
subjects whose personal qualities are unknown to their leaders.

(Smith and Smith 2000:462)
By contrast, the analysts report that Labours manifesto is dominated by photos and collages, featuring leader Tony Blair and a diverse collection of Britons.
According to their interpretation, Labours message is that Great Britain is a land
of happy people sharing life In stark contrast to the Conservatives, Labours is a
populist portfolio. If Labour is to be feared, there is little corroborating evidence
in these pictures of happy families, commuters, and schoolchildren. Visual identification is an important feature of this manifesto (Smith and Smith 2000:463).
As Smith and Smith demonstrate, the non-verbal aspects of a text are just as
important in persuasion as verbal communication. Rhetorical document analysis
encompasses both.
A second qualitative approach to the study of political texts involves discourse analysis (see Eleveld and Filardo Llamas, this volume). Instead of examining the mode of communication, the discourse analyst investigates the broader
values, norms, ideologies, and other contextual factors embedded in a particular
(set of) document(s). Whether from a critical, normative, or empirical perspective, the objective of discourse analysis is to uncover the ideational foundation
underpinning a particular text. The discourse, itself, consists of the language
the bounded range of acceptable terms and statements in which the document
was written. As Louse Phillips described in her examination of Hegemony and
Political Discourse: The Lasting Impact of Thatcherism (1998:851), Discourses
dictate what it is possible to say and not possible to say and thus constrain other
forms of social action. In this sense, the basic premises of discourse analysis
are two-fold: concretely, to elucidate the role of discursive practices in securing
discursive change and hence cultural change; at a more general level, to demonstrate that discourse (language understood as a social practice) is an active
agent in social, political and cultural change (rather than just reflecting change)
(Phillips 1998:848) (see also Berman 2001; Frankenberg 2006:450). This
requires the analyst to investigate, empirically, how the discourse is developed
and propagated through actual instances of language use (Phillips 1998:852,
emphasis in original).
Nigel Copseys Reflections on the ideological evolution of the British National
Party [bnp] (2007) constitute a prime example of discourse analysis. Examining
138 Jared J. Wesley
party manifestos, campaign literature, media interviews, and public speeches, he

assessed the claim that the bnp has transformed its public discourse from one
rooted in 1930s fascism to a more modern form of national-populism. To test this
hypothesis, Copsey drew extensively on the language of British National Party
leader, Nick Griffin, who characterized his own 2005 manifesto Rebuilding
British Democracy as the final and decisive ideological paradigm shift from post
WW2 camouflaged neo-fascism to 21st Century popular nationalism (2007:63).
Copsey noted Griffins abandonment of extremist attacks aimed at specific groups
within British society, in favor of a more tempered assault on the totalitarian,
repressive machinery of the British state (Copsey 2007:73).
Rather than taking Griffin at his word, however, Copsey conducted a systematic analysis of the bnp leaders own statements. Examining Griffins discourse
on a deeper level, he extracts the ideational foundation of the party; what lurks
within is not a commitment to western-style liberal democracy, but a core vision
that represents a fundamental negation of democratic liberalism. That the bnp
appears to be non-totalitarian should not distract us from the fact that it remains
committed to revolutionary rebirth (Copsey 2007:77). According to Copseys
discourse analysis of bnp literature,
the partys new ideological position should be treated with caution. The point that
needs to be made here is that its popular nationalism or national-populism
constitutes a surface ideology that does not change the partys core convictions
The reality is that at the root of the partys ideological modernization is short-term
political expediency: not so much a change of course as an opportune change of
clothing. It is not the transformation from fascism to national-populism but
the recalibration and modernization of fascism itself. It once more testifies to the
almost Darwinian ability of fascism to survive and adapt as an ideology.

(Copsey 2007:7980)
The discourse approach is particularly suited to uncover such shifts in party

ideologies.
Narrative analysis constitutes the third major form of qualitative document
analysis employed by political scientists. Unlike rhetorical analysts (who examine
the delivery of the message) or discourse analysts (whose focus is on the ideas
behind the message), narrative analysts investigate the content, origins, evolution,
and impact of the message as a story about political life. Not all political documents contain narratives, of course. Those that do often present stories in the form
of (un)official histories, myths, legends, folk tales, or personal accounts about the
author or (members of) her political community. These narratives frequently tie
the past to the present and future, speak of political transformations, and identify
specific heroes, villains, and plotlines. These storied messages can be studied
from a variety of perspectives. Some analysts focus on the historical or political

contexts in which the stories are told, for instance, while others examine the role
of the stories in the lives of individual actors or their impact on the community,
as a whole. In this sense, narrative analysts are not primarily concerned with the
factual accuracy of the stories they uncover; rather, their concern is with the ways
in which these stories serve as interpretive lenses through which the authors represent themselves and others.
In addition to their rhetorical analysis, Smith and Smith (2000) examined
the narratives found in British manifestos, comparing the way Conservatives,
Labourites, and Liberal Democrats portrayed the countrys history during the
1997 election campaign. No party waxed nostalgic. No party invoked heroes.
No party relied on sacred national texts. And no party mentioned history prior to
1979, the analysts reported (Smith and Smith 2000:465). Instead, they recount
the recent history that created the disputed state of affairs and the challenges of the
future (Smith and Smith 2000:464). As the governing party, the Conservatives
described the countrys prosperity as the product of their record of unparalleled
success and neo-liberal reforms, for instance. The key to the future, according
to Smith and Smiths interpretation of the Conservative manifesto, is to do more
of the same without knuckling under to those who would turn back (Smith and
Smith 2000:464). The two opposition parties offered a different set of narratives.
The Liberal Democrats lived in a different present, according to the analysts,
one as gray as the landscape of a Welsh slate mine (Smith and Smith 2000:464).
Smith and Smith support this interpretation by quoting directly from the LibDem manifesto, which suggested Eighteen years of Conservative government
have left our society divided, our public services run down, our sense of community fractured, and our economy under-performing (Smith and Smith 2000:464).
By contrast, Tony Blairs Labour platform was more pragmatic than idealistic. I
believe in Britain, he says to begin the manifesto. It is a great country with a great
history. The British people are a great people. But, he continues meaningfully, I
believe Britain can and must be better (Smith and Smith 2000:464). Interestingly,
according to the analysts, Blairs vision for the future marked a sharp break form
Labours historic narrative (Smith and Smith 2000:465). Quoting Blair, they
noted the Labour leaders
aim to put behind us the bitter political struggles of left and right that have torn
our country apart for too many decades. Many of these conflicts have no relevance whatsoever to the modern world public versus private, bosses versus
workers, middle class versus working class. It is time for this country to move on
and move forward. We are proud of our history, proud of what we have achieved
but we must learn from our history, not be chained to it (p. 2).

(Smith and Smith 2000:465)
140 Jared J. Wesley
The narrative approach enabled the identification of the three different stories,
narratives, from the three political parties.
This list of qualitative approaches is by no means exhaustive. Political documents are ripe for a variety of other inquiries. Philosophers may find them useful
sources for hermeneutical examination, political scientists for party positioning,
linguists for semiotic analysis, historians for process tracing, and critical analysts
for deconstruction. Nor are rhetorical, discourse, and narrative analyses necessarily exclusive. Researchers will often combine these modes in a single study
(see: Smith and Smith 2000). Furthermore, as other contributors to this volume
demonstrate, concepts like rhetoric (Dahlberg and Sahlberg, Gryc and Moilanen,
Leeuwen, Maks), discourse (Boyd, Montesano Montessori), and narrative may be
analyzed quantitatively. Rather, the intent of the foregoing discussion is to highlight three of the most common approaches to the study of political documents, as
a means of identifying the sorts of themes uncovered during the qualitative coding process. The main concern of this chapter is to ensure that the uncovering of
themes through a qualitative process meets certain standards of empirical rigour.
After a discussion of the ontological foundations of the quantitative and qualitative
traditions, this chapter establishes a set of guidelines for the trustworthy practice
of qualitative textual analysis.
Building bridges
Three ontological perspectives
The following discussion proceeds under the assumption that the qualitative and
quantitative approaches to textual analysis are commensurable under the same,
general standards of empirical inquiry. While distinct in many ways, each tradition contributes its own set of tools for the overall toolkit of the twenty-first
century political scientist (Wesley 2011c). This view is only one of three general
perspectives on the relationship between quantitative and qualitative methods
(Bryman 2001:276; Corbetta 2003:50). While ideal types, in that no researcher is
likely to adhere entirely or permanently to one set of beliefs, the distinctions are
informative for document analysts and others who engage in research on either
side of the quantitative-qualitative divide.
The first perspective holds that the quantitative and qualitative traditions are
so ontologically distinct as to be incommensurable. Scholars in this purist school
believe in a hard-and-fast connection between quantitative methods and the tenets
of positivism,1 on one hand, and qualitative methods and interpretivism, on the

other. According to this perspective, quantitative positivists believe in the principles of inherency and verifiability, which puts them at odds with the belief among
qualitative relativists that all reality is socially constructed. As Manheim et al.
(2002:318) describe, Some quantitatively oriented scholars regard at least some
qualitative work as so dependent on the perceptions of the individual researcher
and so focused on specific cases as to be unverifiable and essentially useless. In
contrast, some qualitatively oriented scholars judge quantitative methods to be so
incomplete in their representation of reality as to be empirically misleading In
this environment, researchers toil in opposing camps either parallel, but separate, in their pursuit of knowledge, or actively seeking to undermine the other.
Political science has not been immune to the above tensions; thankfully, however, Most empirical researchers work primarily with either qualitative or quantitative methods but can see value in the other approach (Manheim, Rich, and Willnat
2002:318). This second perspective is embodied in the work of King, Keohane and
Verba (1993), and holds that both quantitative and qualitative research methods
are commensurable under the positivist approach to social life. In their words,
the differences between the quantitative and qualitative traditions are only stylistic
and are methodologically and substantively unimportant. All good research can
be understood indeed is best understood to derive from the same underlying
logic of inference. Both quantitative and qualitative research can be systematic and
scientific, provided each submits to the rules of scientific inference rules that
are sometimes more clearly stated in the style of quantitative research (1993:45,
6). Critics of the KKV approach accuse the authors of developing a quantitative template for qualitative research a premise that presupposes the superiority
of the former over the latter (Brady, Collier, and Seawright 2004:3). For this reason, qualitative purists have anointed King et al. as headmasters of the quantitative imperialist school, imposing positivist concepts like hypothesis-testing and
inter-subjectivity on an unwilling qualitative community. In fairness to King et
al., their aim was to bridge the quantitative-systematic-generalizing / qualitative-humanistic-discursive divide (King, Keohane, and Verba 1993:4). Less pejoratively, then, one may refer to theirs as the neo-positivist perspective.
1. For the purposes of this discussion, positivism is defined by the following three tenets:
(1)scientific methods (i.e., the testing of hypotheses derived from pre-existing theories) may be
applied to the study of social life; (2) knowledge is only generated through observation (empiricism); and (3) facts and values are distinct, thus making objective inquiry possible (Snape and
Spencer 2006).
142 Jared J. Wesley
A third perspective takes a middling view of the relationship between quantitative and qualitative methods. Developed most coherently in a volume edited
by Brady, Collier and Seawright (2004), the dualist school promotes the coexistence of quantitative and qualitative traditions within a broad social scientific
enterprise. Unlike purists, dualists see value in collaboration between quantitative and qualitative researchers, and an important element of interdependence
in their relationship. Compared to neo-positivism, the dualist school sees
strengths and weaknesses in both approaches. As Brady et al. (2004:10) put it,
In the social sciences, qualitative research is hard to do well. Quantitative research
is also hard to do well. Each tradition can and should learn from the other. One
version of conventional wisdom holds that achieving analytic rigor is more difficult in qualitative than in quantitative research. Yet in quantitative research,
making valid inferences about complex political processes on the basis of observational data is likewise extremely difficult. There are no quick and easy recipes
for either qualitative or quantitative analysis. In the face of these shared challenges, the two traditions have developed distinctive and complementary tools
(emphasis in original).
(Brady et al. 2004:10)
Instead of struggling for methodological supremacy, dualists implore all social

scientists to refine and develop the battery of techniques on offer, and above
all to be as explicit as possible about the implications of the methodologies we
employ (Laver 2001:9).
While acknowledging that many readers view the world from the purist and
neo-positivist perspectives, the following discussion proceeds along dualist
lines. According to this view, social science is the systematic study of the social
world; the definition of what constitutes systematic is contentious, a debate that
is explored in greater detail below.
Two traditions of document analysis

As the paradigm governing most quantitative research, positivism elevates two key
concepts as crucial elements of any legitimate study in political science: validity
and reliability. The former term refers to the importance of ensuring that ones
findings accurately represent the concepts under examination. In content analysis,
for example, a valid conclusion about the positive tone of a particular document must incorporate evidence of the authors optimism, cheerfulness, sanguinity, buoyancy, exuberance, or other senses of approbation. From a quantitative
perspective, this evidence is often derived by counting the number of positive
references, be they measured in terms of keyword mentions, phrases, sentences,
paragraphs, or other units of analysis. Reliability, on the other hand, refers to the
consistency of a particular measurement the extent to which a particular assessment would yield identical results if repeated under the same conditions. Content
analysts typically measure this consistency through inter-coder reliability testing,
a process in which the analysts measurements are checked against those of an
independent researcher.
As standards of academic rigour, both validity and reliability are rooted in
the assumption that the information contained in documents is inherent that
the evidence embedded in the text is objectively identifiable. Armed with a list
of pre-defined variables, the content analysts task is to mine the documents in
search of specific bits of data. This information is then analyzed, statistically, to
discern important patterns existing within and between the documents.
While requiring a similar level of precision, the qualitative approach differs
from quantitative content analysis in important ways. Rather than viewing data as
inherent to the documents, themselves, most qda researchers reject the notion of
inter-subjectivity. From this perspective, the meanings invoked by texts need not
be shared in a direct sense (Krippendorff 2004:2223, emphasis in original) (see
also: Morse and Richards 2002:125). To many qda researchers, their particular
interpretation of the text is just one of many possible readings, thus imposing
a different set of methodological burdens on them as they seek to convince their
readers of the persuasiveness of their analyses (Gerring 1998:298; Laitin 1986:13).
As Manheim et al. (2002:317) point out,
Quantitative researchers are usually able to employ some well-established rules
of analysis in deciding what is valid evidence for or against their theory. These
include such tools as measures of statistical significance and statistical tests of
validity, as well as formal logic. Qualitative researchers generally lack this type of
commonly agreed to and objective tool. Rather, they must rely on their ability to
present a clear description, offer a convincing analysis, and make a strong argument for their interpretation to establish the value of their conclusions. Advocates
of qualitative methods argue that this is an inevitable result of seeking to deal with
the richness of complex realities rather than abstracting artificially constructed
pieces of those realities for quantitative analysis. Critics of their approach contend
that the vagueness and situational nature of their standards of evidence make it
difficult (if not impossible) to achieve scientific consensus and, therefore, to make
progress through cumulative knowledge.
(Manheim et al. 2002:317)
One particularly stinging critique holds that the findings of most qualitative analyses tend to be conjectural, non-verifiable, non-cumulative, meanings arrived
at by sheer intuition and individual guesswork (Cohen 1974:5). In short, qualitative researchers are subject to the criticism that they leave their readers with
little choice but to trust that their interpretations of the data are accurate and
legitimate.
144 Jared J. Wesley
Trustworthiness in political science

To guard against these criticisms, disciplinary standards require all political scientists to adhere to certain rules when it comes to treating texts as data. In particular, both quantitative content analysts and qualitative document analysts must
establish the legitimacy of their research by protecting its trustworthiness in the
eyes of the broader academic community.
The notion of trustworthiness is borrowed from the seminal research of
Egon Guba and Yvonna Lincoln (1985). According to their dualist view, while
quantitative research tends to be conducted under the premises of positivism,
and qualitative inquiry under the auspices of interpretivism, there is considerable
middle ground upon which to build consensus over the norms of social scientific
analysis. Whereas scholars working in the qualitative tradition tend to reject the
objectivity embedded in concepts like validity and reliability, for instance, Guba
and Lincoln found that they tended to value and impose a similar set of standards
on their own work. Their book, Naturalistic Inquiry, served as a sort of Rosetta
Stone for interpreting four such common norms.
First, according to Lincoln and Gubas advice, all document analysts must
protect the authenticity or truth value of their research. An authentic analysis is one that offers a genuine interpretation of reality, or an accurate reading of
a particular (set of) document(s). This is referred to as measurement validity
in the quantitative-positivist tradition, and credibility in the qualitative-interpretivist tradition. For the latter, the objective is less to offer a truthful account
of the information found in the document, than to provide a believable interpretation of the meanings found therein (Richerson and Boyd 2004:410411).
The authenticity of a qualitative analysis, then, relies upon the subjective evaluation of the reader, as opposed to being based against some objective standard
(Krippendorff 2004:314).
Portability is a second concern for analysts dealing with political documents.
To make a substantive contribution to knowledge, most social scientists concur
that their inquiries must offer insights extending beyond the specific cases under
study (Bryman 2004:539). In quantitative-positivist terms, this is referred to as
external validity the generalizability of a particular analysis to broader questions about political life. Content analysts strive to convince their readers that
their findings can be expanded to other documents, from other sources, times,
or places, for instance. The term transferability is preferred among those conducting qda, once again reflecting their reluctance to accept the inter-subjectivity
of their interpretations. Rather than establishing the generalizability of their analyses through tests of statistical significance, for example, qualitative document analysts rely upon their readers to assess the broader applicability of the lessons drawn
from their findings. In this sense, the question of whether the results of a qualitative document analysis can be extended to another context must be answered
not by the original investigator but by the student seeking to make the transfer
(Lewis and Ritchie 2006:145; Merriam 2002:228229).
Third, researchers studying political documents must be wary of the precision
of their analyses. Discussed above, content analysts tend to assess this aspect of
trustworthiness in terms of reliability, through inter-coder testing. While replicability is fundamental to the positivist approach, however, its relevance is more
contentious in the interpretivist tradition. As a consequence, many qualitative
document analysts use the term dependability to describe the precision of their
research. This captures the belief that, provided the research is conducted in a
transparent manner, readers may assess the accuracy of the findings by asking,
Would I have reached the same general conclusions, given the opportunity to
read the same set of documents under similar conditions? An affirmative answer
would confirm the dependability of the analysis.
The fourth and final concern among document analysts surrounds the impartiality of their observations. Social science is premised on the capacity of its practitioners to produce relatively unprejudiced knowledge about the social world,
through findings that are reflective of reality as opposed to their own pre-determined beliefs (Marshall and Rossman 1989:147). In quantitative research, this
means preserving the objectivity of the analysis. Because they are more likely to
consider personal biases to be unavoidable, if unfortunate, factors in the research
process (King, Keohane, and Verba 1993:1415; Merriam 2002:5), qualitative
document analysts tend to acknowledge (even embrace) the subjectivity of their
interpretations. To remain impartial, they must achieve confirmability in their
findings, ensuring that their conclusions are drawn from the evidence at hand,
as opposed to the predispositions of the researcher. The results of a qda study
are confirmable if the inferences drawn are traceable to data contained in the
documents, themselves, and if the preponderance of evidence corroborates those
findings. This is the very essence of empirical inquiry.
The foregoing discussion suggests that, relative to quantitative content analysis, qda tends to place a heavier burden on the reader of the study to assess
its trustworthiness. Some argue that this places too little responsibility on the
researcher to defend the merits of the analysis. This criticism is misplaced, for in
order to convince their audience of the trustworthiness of their research, qualitative data analysts must take equal care to meet the following disciplinary expectations of their work.
146 Jared J. Wesley
Achieving trustworthiness in qualitative document analysis

According to Guba and Lincoln (1994) the four norms of trustworthiness can
be assured by (1) being explicit about the process by which the evidence is interpreted, and by (2) providing access to ones data so that findings may be verified.
While providing for a post-hoc verification of the authenticity, portability, precision, and impartiality of their analyses, these two general safeguards are not sufficient to assure the trustworthiness of analyses. As Morse et al. (2002:14) argue,
by paying attention to the end of the study rather than the conduct of the research
itself, investigators risk missing serious errors until it is too late to correct them.
According to their assessment,
in the time since Guba and Lincoln developed their criteria for trustworthiness,
there has been a tendency for qualitative researchers to focus on the tangible
outcomes of the research (which can be cited at the end of a study) rather than
demonstrating how verification strategies were used to shape and direct the
research during its development. While strategies of trustworthiness may be useful in attempting to evaluate rigor, they do not in themselves ensure rigor. While
standards are useful for evaluating relevance and utility, they do not in themselves
ensure that the research will be relevant and useful.

(Morse et al. 2002:16, emphasis in original)
Heeding Morse et al.s advice, the following discussion provides a series of guidelines for the conduct of qualitative document analysis which are more specific
than the first key assurance of Guba and Lincoln. Their compilation may create
a useful checklist for reviewers, but their greater value lies in the support they
provide for ensuring the trustworthiness of document analysis, be it quantitative
or qualitative. Four sets of concerns are outlined, including those dealing with
(1) triangulation, (2) intense exposure and thick description, (3) audit trails and
discrepant evidence, and (4) intra- and inter-coder testing.
Triangulation
When using any form of data, political scientists are wise to corroborate their
findings using other types and sources of evidence. Document analysts are no
different, in this respect (Boyatzis 1998:xiii). Whether combining their findings with interviews, focus groups, or other research strategies, or conducting
a mixed-methods form of research involving both quantitative and qualitative
forms of textual analysis, researchers using political documents as their primary
sources of evidence must substantiate their findings with some form of external
support (Tashakkori and Teddlie 2003:x). For qualitative document analysts, this
triangulation may take several forms.
The first, and most common, involves quantizing ones findings. This means
buttressing any subjective, qualitative interpretations of the latent elements of a
text with more objective, quantitative analyses of its manifest content (Hesse-Biber
and Leavy 2006:326330). References to the existence of a particular theme in a
set of documents, for instance, may benefit from an indication of how many times
a particular set of keywords appeared in the texts. Doing so bolsters (the readers
confidence in) the precision of the analysis.
John Gerring applied this technique in his study of Party Ideologies in America
(1998). Confronted with the choice between quantitative content analysis and
qualitative document analysis, Gerring opted for the latter. To make claims about
party ideologies, he argued, one must involve oneself in the meat and gristle of
political life, which is to say in language. Language connotes the raw data of most
studies of how people think about politics, for it is through language that politics
is experienced (Gerring 1998:298). In this vein, Gerring (1998:297) suggested, it
would be unrealistic to expect content analysis to bear the entire burden of analysis on a subject as vast and complex as party ideology. To begin with, one would be
forced to scale back the quantity of evidence in a fairly drastic fashion Second,
and perhaps more significantly, content analysis is somewhat less scientific than
it appears. Since the meaning of terms is not static or univocal, words do not fall
automatically within content analysis categories.
In search of this language, Gerring turned to American party platforms,
dating back to 1828. There, he found distinct rhetorical patterns, such that WhigRepublicans spoke in terms of order versus anarchy and the state versus the
individual and the Democrats in terms of liberty versus tyranny and the
people versus the interests. To substantiate his interpretation, Gerring provided
detailed paraphrasing and copious quotations from the party platforms. Yet he
also bolstered this analysis with a content analysis of specific terms, phrases, and
concepts used by the various parties. By including graphs depicting the differentiated use of words like liberty or the people, over time, Gerring effectively
quantified his qualitative findings.
A second method of triangulation involves member-checking a familiar
tool to those conducting field or focus group researchers. These analysts often
verify the results of their observations with the subjects, themselves, as a means
of verifying the authenticity of their findings. In document analysis, this means
consulting the authors of the texts, to see if ones interpretations match their original motives or intent. It may not be possible to consult the author of a specific
document, whether due to death, anonymity, location, or disinclination. Where
148 Jared J. Wesley
possible, however, member-checking should be viewed as a valuable, continuous

process during data analysis [not simply] as verification of the overall results
(Morse et al. 2002:16).
There may be disagreement between the researcher and the author, of course;
indeed, there is often a healthy tension within research conducted from the emic
and etic perspectives. In this sense, an authors intent may not be conveyed
effectively to his or her audience, in which case the researchers interpretation
may provide a more authentic account of the readers view of a particular text.
Moreover, given that qda is conducted with the understanding that no two readers are likely to come to identical interpretations of a given text, some disagreement between the author and the researcher is to be expected. The objective of
member-checking is to provide at least some safeguard as to the authenticity of
the researchers interpretation of the text. Any disjunction between those findings and the authors own interpretation should not be taken necessarily as a
refutation of the former, but rather a point to be explored during the analysis.
The analyst should be prepared to defend his or her interpretation as the more
trustworthy account of the text.
Intense exposure and thick description

A second set of guidelines requires qualitative document analysts to immerse
themselves in their texts and produce detailed accounts of their findings. Some
people refer to this process as one of soaking and poking, although the imagery
belittles the amount of rigour involved (King, Keohane, and Verba 1993:3643;
Putnam 1993:12; Shively 1998:17). Granted, like any researcher, document analysts ought to marinate in their data until no new, alternative interpretations
appear to emerge. (This is often referred to as the saturation point.) Yet, to offer a
trustworthy and systematic account, this process must be analytical and empirical,
not simply a matter of osmosis.
Among political scientists, the coding process, itself, is one of the least-examined elements of qualitative document analysis. As health researchers Hsiu-Fang
Hsieh and Sarah E. Shannon (2005) describe, there are three general techniques
of qualitative textual coding, the choice among which depends largely upon the
research question at hand. The first type of conventional document analysis is
employed in exploratory studies, where existing theories or data in the subject area
are limited. Under these circumstances, analysts cannot rely on previous research
as a guide, but must allow the themes to emerge from the texts, themselves.
In engaging in this conventional, inductive style of inquiry, analysts typically
rely on a systematic three-stage coding process (Miles and Huberman 1994).
During the first open coding phase, the researcher selects a small sample of the
documents for a preliminary, in-depth review. She makes general notes about
the broad themes that characterize each document individually, and all texts collectively. These themes are knitted together during a second stage of axial coding, in which all documents are consulted. Patterns are given specific labels,
and certain passages are tagged as belonging to one or more categories (see:
Boyatzis 1998:31). A third stage of selective coding involves checking and rechecking these tags, ensuring that labels are applied properly and noting any discrepant evidence (see: Creswell 1998:150152; David and Sutton 2004:203212;
Morse and Richards 2002:111128; Neuman and Robson 2007:337342; Punch
2005:199204).
Consider a recent study of party platforms in the three Canadian Prairie
Provinces of Alberta, Saskatchewan, and Manitoba (Wesley 2011b). Like Gerring,
the researcher was motivated by the desire to uncover patterns in the rhetoric of
dominant parties in each province. His specific intent was to discern whether the
different political cultures found across the region were connected, in some way,
to the elite level discourse taking place during election campaigns. Was Albertas
conservative political culture associated with the rhetoric of its dominant politicians? Did Saskatchewan politicians campaign with a left-wing accent, matching
the social democratic ethos of the province? And was Manitobas political culture
of moderation connected to the campaign messages of its leading parties?
Armed with these initial questions and hypotheses, the analyst entered the
inquiry by assuming that no such patterns existed, seeking evidence to support
their presence and reject the null hypothesis. In search of these themes, he collected and analyzed over eight hundred pieces of campaign literature, dating back
to 1932.2 During the open-coding stage, the analyst detected certain broad-ranging themes. In Alberta, the discourse appeared to revolve around notions of liberty and anti-conformity, whereas parties in Saskatchewan tended to campaign
on the importance of solidarity and community, and Manitoba on tolerance and
modesty. These observations were recorded in the form of memos, which were
used to direct the second phase of axial coding.
During this second stage, key passages were highlighted as belonging under
the broad categories identified during the first read-through. Statements in
Alberta were tagged as belonging under the category of freedom, including subthemes like populism, individualism, and provincial autonomy. In Saskatchewan,
axial coding classified certain statements as being evidence of the provinces
2. Many of these documents are available online, as part of the Poltext Project collection.
150 Jared J. Wesley
security-based discourse, including references to collectivism, strong government, and polarization. Manitoba party platforms were coded for evidence of
moderation (incrementalism, pragmatism, and a loose form of partisanship).
Throughout this process, the analyst was on constant guard for discrepant evidence, including themes that may have appeared across provincial borders.
A third, systematic pass through the documents involved significant reflection
and revision. Some passages were reassigned to different categories, for instance,
and numerous cases were highlighted that challenged the tidiness of the earlier
analysis. Some Alberta politicians made use of social democratic rhetoric, for
instance, while parties in Saskatchewan and Manitoba occasionally invoked conservative terminology during campaigns. This discrepant evidence was recorded,
reported, and addressed in the final report (see below). Through this three-stage
process, the analyst was able to systematically identify several themes, refine their
content, and support their existence with evidence drawn from the documents,
themselves.
Hsieh and Shannon label a second, more structured technique as the directed
mode of qualitative textual analysis. In this approach, the researcher is able to rely
on previous research for direction when interpreting the content of various documents. This prior knowledge may be used to conceptualize the research question,
or to refine the coding scheme prior to analysis. In other words, Using existing
theory or prior research, researchers begin by identifying key concepts or variables
as initial coding categories Next, operational definitions for each category are
determined using the theory (Hsieh and Shannon 2005:1281). Armed with this
scheme, the researcher then reviews the documents, coding separate passages as
matching various pre-determined themes and recording any discrepant evidence.
Depending on the fitness of the data to the coding scheme, categories may require
deletion or revision, and additional themes may be identified.
Consider Michelle Weinroths (2004) qualitative study of the Canadian Liberal
Partys marketing strategy in its Anti-Deficit Campaign during the mid-1990s.
Her approach was a combination of rhetorical, discourse, and narrative analysis,
as she sought to account for the success of the Liberal Party in maintaining its
popularity amidst massive cutbacks in social services. The questions that I pose,
she wrote, reflect a line of inquiry addressed by several critics of neo-liberalism
(Weinroth 2004:46). Many of these researchers focused on the economic and fiscal discourse generated by a nexus of corporate, academic, media, and political elites. Using this literature to contextualize and conceptualize her own study,
Weinroth (2004:46) expanded upon earlier findings through her own unique
interpretation of the data:
While my own explanatory model shares much with these studies, its distinctiveness lies in its conceptual framework, for it stresses, unlike these other treatises,
that the Liberal campaign deployed a symbolic language of nationalism in fiscal
form, and that the mystique and persuasive power of such an ideology resides in
its archetypally dramatic patternMy essay turns its attention to the theatrical
and nationalist dimensions of such propaganda in an effort to show the workings
of a key episode in Canadian consensus-building (emphasis added).

(Weinroth 2004:46)
A third qualitative technique identified by Hsieh and Shannon is known as summative content analysis. Instead of examining the texts in a more holistic sense,
the researcher explores the different meanings attached to specific concepts found
therein. After searching the documents for manifest content that is, the appearance of the term(s) under study the investigator probes the different contexts in
which the particular words were employed. Did certain types of authors use the
term(s) in similar ways, for example, or has the usage of the concept changed over
time? After reviewing the documents in search of responses to these types of questions, the researcher returns to the text to classify each appearance of the term(s)
as belonging to one or several categories.
Roderick P. Hart and his colleagues used summative content analysis in their
investigation of The American People in Crisis (2002). Drawing on the rhetoric
contained in Congressional speeches surrounding the events of September 11th
and the Clinton Impeachment proceedings, the researchers examined the use of
the term American people during periods of national turmoil. Hart et al. found
that politicians referred to the citizenry in a variety of different ways, which, upon
further examination and using previous literature as guidance, they divided into
six main categories (the Peoples Time, the Peoples Situation, the Peoples Role, the
Peoples Actions, the Peoples Qualities, and the Peoples Opponents).
When applying any of the three techniques described by Hsieh and Shannon,
researchers must be meticulous in reporting the results of their analysis, and the
evidence upon which the interpretations were based. Known famously as thick
description, Gerring refers to this process as grounding ones findings
in copious quotations from the principals. At times, this may seem laborious.
However the inclusion of actual language in a rhetoric-centred study should be
seen as equivalent to the inclusion of raw data in a qualitative study; both allow
the reader to evaluate the evidence without relying entirely on the authors own
authority. It also provides a depth otherwise lacking in discussions of abstract
concepts and content-analysis statistics.
(Gerring 1998:298)
152 Jared J. Wesley
This begs the question, however: How much data is enough to substantiate ones
findings? Without enough supporting evidence, a qualitative document analysis amounts to little more than armchair interpretation. Even with proper citations, too much paraphrasing may lead readers to question the authenticity and
impartiality of the study. Conversely, How much data is too much? Without the
researchers own voice, a study consisting of too many direct quotations amounts
to transcription, not inquiry (Morse and Richards 2002:188). Striking a balance
between evidence and analysis is especially challenging for qda researchers, in this
regard (Platt 2006:111112).
While no disciplinary convention exists, as a safe rule of thumb, Berg
(2004:270) recommends including at least three pieces of corroborating evidence
for each major interpretation. These may come in the form of direct quotations
or detailed paraphrases, and depending upon the researchers style and chosen
venue can be incorporated in-text or by way of footnote. This places the onus on
the analyst to support his or her interpretation with adequate evidence, while providing the reader with the assurance that the findings are not derived arbitrarily.
Audit trails and discrepant evidence

In order to provide readers with the opportunity to assess the authenticity and
precision of their analyses, researchers must also report the exact process through
which they achieved their results. In quantitative analysis, this is most efficiently
accomplished through the publication of the research instrument (questionnaire,
coding manual, or other guides). With no standardized instrument, qualitative
document analysis requires that its practitioners provide detailed accounts, not
only of their findings, but of the process by which they reached their conclusions
(Platt 2006:116). This entails creating an audit trail and reporting any discrepant
evidence that may challenge their interpretations (Altheide 1996:2533).
As Holliday (2007:7) suggests, most qualitative research involves making
sense of the often messy reality of social life. Doing so requires the qualitative
document analyst to make dozens of difficult and subjective decisions throughout
the research process choices about which similarities and differences constitute real patterns in the text; to what degree certain parallels constitute genuine
themes; which titles should be used to identify these themes; which passages
constitute solid evidence; how much discrepant evidence must exist to refute a
particular set of findings; and many others. There are no inherently right or wrong
answers to such questions; there are only stronger or weaker justifications of these
choices. As a result, qualitative document analyst must be explicit in identifying
and defending the various decisions they made throughout the research process.
Contrary to most methods textbooks, an audit trail is not constructed solely

at the end of the study. While a post-hoc report may satisfy the needs of reviewers,
documenting the development of a completed analysis does little to ensure that the
study, itself, is conducted in a trustworthy fashion (Morse et al. 2002:16). Rather,
researchers must keep detailed records of their progress throughout the data gathering, analysis, and reporting stages. These notes are often kept in the form of
memos or journals, and serve two objectives. For the benefit of the reader, they
allow the analyst to more accurately report the outcome and rationale behind the
various decisions made. Second, the process of chronicling, itself, serves a valuable
purpose, as it ensures the analyst is aware of, and continuously seeking to justify,
the many choices made throughout the inquiry.
Some of the most important decisions concern how to deal, and whether to
report, discrepant evidence. By including only information that serves to confirm
their interpretations of the text, qualitative data analysts often face criticism for
offering analyses that are too tidy or circular. On the latter, critics of some
qda studies cite the researchers for entering the analysis with pre-defined hypotheses that, in turn, determine what they see as significant (George 2006:155). To
avoid succumbing to this tendency, Becker (1998), Esterberg (2002:175), and Berg
(2004:184) recommend employing the null hypothesis trick, by which the analyst enters the inquiry assuming that no patterns exist; he or she must then assemble evidence from the documents to establish the existence of any themes. This
helps shift the burden of proof onto the researcher, and away from the reader.
Of course, qualitative document analysts need not prove the truth of their
interpretations beyond all doubt. Most social scientists operate on a different standard of trustworthiness, requiring their peers to establish the persuasiveness of
their findings against competing interpretations. In quantitative research, persuasiveness is often measured in terms of probability (e.g., statistical significance),
whereas qualitative researchers often speak in terms of plausibility.3 Each term
connotes a distinct, but related, standard of legitimacy. As Richerson and Boyd
(2004:410411) note, plausibility arguments have three features in common
with more conventional hypotheses developed under the positivist paradigm:
3. As a method of scientific explanation, plausibility arguments are well established in both the
natural and social sciences. Indeed, plausibility arguments underlie much of what we know
about the physical and social world; they underpin many of the theories and laws developed by
mathematicians, physicists, archaeologists, evolutionary biologists, anthropologists, and others. Some plausibility arguments are used in exploratory research, developing hypotheses to be
tested empirically in the future. Others are untestable, at least given current knowledge, technology, theory, or conditions. (Consider theories surrounding the existence of the quark, for
instance.) In cases like this, plausibility arguments often resist proof or even testing in
the formal sense.
154 Jared J. Wesley
(1) a claim of deductive soundness, of in-principle logical sufficiency to explain

a body of data; (2) sufficient support from the existing body of empirical data to
suggest that it might actually be able to explain a body of data as well as or better
than competing plausibility arguments; and (3) a program of research that might
distinguish between the claims of competing plausibility arguments. The differences are that competing plausibility arguments (1) are seldom mutually exclusive, (2) can seldom be rejected by a single sharp experimental test (or small set of
them), and (3) often end up being revised, limited in their generality or domain of
applicability, or combined with competing arguments rather than being rejected.
In other words, competing plausibility arguments are based on the claims that
a different set of submodels is needed to achieve a given degree of realism and
generality or that a given model is correct as far as it goes, but applies with less
generality, realism, or predictive power than its proponents claim.

(Richerson and Boyd 2004:410411)
Thus, when developing their interpretations, qualitative document analysts need

not feel pressure to prove their reading is the only accurate one. In fact, they are
encouraged to report evidence that places reasonable bounds on their findings.
An accomplished qda researcher
considers not just one inferential hypothesis when reading and rereading the original communication material, but also many alternatives to it. He systematically
weighs the evidence available for and against each of these alternative inferences.
Thus, the results of his analysis, if fully explicated, state not merely (1) the favored
inference and the content evidence for it, but also (2) alternative explanations of
that content evidence, (3) other content evidence which may support alternative inferences, and (4) reasons for considering one inferential hypothesis more
plausible than others.
(George 2006:155)
Doing so, and reporting the specific decisions in the audit trail, boosts the credibility of the analysis (Holliday 2007:167181).
These three sets of guidelines, triangulation, intense exposure and thick
description, and audit trails and discrepant evidence, provide a more detailed
account of the first key assurance of Guba and Lincoln, making the process of
interpretation more explicit. Their second key assurance is equally important,
though. Qualitative document analysts ought to provide reasonable access to
their raw materials. This is not simply as a courtesy to reviewers, or to protect
against charges of inauthenticity, imprecision, or partiality. It is also crucial to the
advancement of knowledge, as it permits other researchers to conduct their own
inquiries without having to undergo the same painstaking process of collecting the
raw materials. While not always possible (due to resource constraints or concerns
over copyright or confidentiality), ideally documents should be placed in the public domain. Given advances in digital scanning and optical character recognition,
it is becoming increasingly easier to post texts online, in electronic form. One such
collection has been amassed under the auspices of the Poltext Project (see also
Collette and Petry, this volume).
Funded by a grant from the Fonds qubcois de la recherche sur la socit
et la culture, and housed at Universit Laval in Quebec City (and online at www.
poltext.org), the Poltext project collects and provides access to political documents
drawn from across Canada and over time.4 Amassed by a research team from
across the country, the open-source collection is one of the largest of its kind
in North America. It contains a growing assortment of party platforms, throne
speeches, budget speeches, and a variety of other political documents at both the
federal and provincial levels, dating back to the 1960s. As such, the Poltext collection serves as an unparalleled source of data on democratic life in Canada. Of note,
access to provincial-level data is especially valuable to comparative researchers,
both in Canada and beyond. The ten Canadian provinces constitute an underused
series of laboratories for the comparative study of public policy, party ideology,
political rhetoric, and many other areas of political science research.
The main advantage of the Poltext project lies in its provision of raw textual
data for both quantitative content analysts and qda researchers. Unlike similar
databases, the information found in the Poltext collection does not come as
pre-packaged data. While useful, one of the drawbacks to many other manifesto collections lies in the fact that their users must conform their research
questions and methods to the data; the resulting inquiries amount to secondary
data analysis, rather than primary research. Recent changes to the Comparative
Manifesto Project Database have made its collection free-access, as well.
Finally, to guard against partiality, qualitative document analysts ought to
investigate and report their personal biases. Qda may be considered a form
of unobtrusive research, in that it does not directly involve human subjects.
However, as the researchers are key instruments of qualitative research filtering the raw documents through their own personal lenses in order to produce
data it is important to investigate possible sources of contamination (Merriam
2002:5). Thus analysts must undertake a process of critical self-reflectionbefore
and during the inquiry, and disclose the results as part of the final report
(Creswell 2003:182).
4. The use of data from the project for publication purposes is subject to the mention of the
following source: Poltext project (www.poltext.org) Universit Laval (Qubec). The Poltext
project is funded by a grant from the Fonds qubcois de la recherche sur la socit et la culture.
156 Jared J. Wesley
Conclusion
To reiterate, the foregoing discussion serves as a set of guidelines for the conduct
of trustworthy qualitative document analysis. This is by no means an exhaustive
list (see Wesley 2009, 2011a). Nor is it intended as a checklist for evaluating the
authenticity, precision, portability, or impartiality of qda studies. Morse et al.
(2002:16) are correct:
Using standards for the purpose of post-hoc evaluation is to determine the extent
to which the reviewers have confidence in the researchers competence in conducting research following established norms. Rigor is supported by tangible evidence using audit trails, member checks, memos, and so forth. If the evaluation is
positive, one assumes that the study was rigorous. We challenge this assumption
and suggest that these processes have little to do with the actual attainment of
reliability and validity. Contrary to current practices, rigor does not rely on special
procedures external to the research process itself.
(Morse et al. 2002:16)
Often dismissed as simply reading, reviewing, or interpreting texts, trustworthy qualitative analysis of political documents requires as much rigour as any
other methodology. Scholars who employ qda will encounter this reality firsthand when presenting their work for review by quantitatively-minded audiences.
Reviewers will often ask, are we simply expected to trust you, that your interpretation of these texts is valid and reliable? Or, who is to say your reading of these
materials is most accurate? While it is true that trusting the evidence provided
in qda studies is comparable to trusting the data presented in regression tables
and other forms of quantitative presentation in that the reader will seldom refer
back to the source material to replicate the analysis themselves such a response
is unlikely to persuade ones critics. Instead, qda scholars must incorporate checks
and balances into their analyses, to demonstrate how their reading of the texts is
as trustworthy as anyone elses.
The tremendous advances made in quantitative content analysis in recent decades have dwarfed the development of similar innovations in qda. Yet, qualitative
document analysis remains a core methodology in political science research. The
foregoing discussion serves as a baseline for similar progress in qualitative document analysis, and encourages students to further explore ways of improving the
accuracy, scope, reach, and acceptance of qda in the broader political science
community.
References
Altheide, D.L. 1996. Qualitative Media Analysis. Thousand Oaks: Sage.
Becker, H.S. 1998. Tricks of the Trade: How to Think about Your Research While Youre Doing
It. Chicago: University of Chicago Press. DOI: 10.7208/chicago/9780226040998.001.0001
Berg, B.L. 2004. Qualitative Research Methods for the Social Sciences. 5th Edition. Toronto:
Pearson.
Berman, S. 2001. Ideas, norms, and culture in political analysis: Review article. Comparative
Politics 33(2), pp. 231250. DOI: 10.2307/422380
Boyatzis, R.E. 1998. Transforming Qualitative Information: Thematic Analysis and Code Development. Thousand Oaks: Sage Publications.
Brady, H.E., D. Collier and J. Seawright. 2004. Refocusing the discussion of methodology. In
H.E. Brady and D. Collier (eds.) Rethinking Social Inquiry: Diverse Tools, Shared Standards.
Boulder: Rowman & Littlefield, pp. 320.
Bryman, A. 2001. Social Research Methods. Toronto: Oxford University Press.
Bryman, A. 2004. Social Research Methods. 2nd Edition. Toronto: Oxford University Press.
Cohen, A. 1974. Two-Dimensional Man: An Essay on the Anthropology of Power and Symbolism
in Complex Society. Berkeley: University of California Press.
Copsey, N. 2007. Changing course or changing clothes? Reflections on the ideological evolution of the British National Party 1999-2006. Patterns of Prejudice 41(1), pp. 6182. DOI:
10.1080/00313220601118777
Corbetta, P. 2003. Social Research: Theory, Methods and Techniques. Thousand Oaks: Sage
Publications.
Creswell, J.W. 1998. Qualitative Inquiry and Research Design: Choosing among Five Traditions.
Thousand Oaks: Sage Publications.
Creswell, J.W. 2003. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches.
David, M. and C.D. Sutton. 2004. Social Research: The Basics. Thousand Oaks: Sage Publications.
Esterberg, K.G. 2002. Qualitative Methods in Social Research. Boston: McGraw Hill.
Frankenberg, G. 2006. Comparing constitutions: Ideas, ideals, and ideologytoward a layered
narrative. International Journal of Constitutional Law 4(3), pp. 439459. DOI: 10.1093/
icon/mol012
George, A.L. 2006. Quantitative and qualitative approaches to content analysis. In J. Scott (ed.)
Documentary Research. Volume 1. Thousand Oaks: Sage Publications. pp. 135160.
Gerring, J. 1998. Party Ideologies in America, 18281996. Cambridge: Cambridge University
Press. DOI: 10.1017/CBO9781139174992
Guba, E.G. and Y.S. Lincoln. 1985. Naturalistic Inquiry. Thousand Oaks: Sage Publications.
Hart, R.P., S.E. Jarvis and E.T. Lim. 2002. The American people in crisis: A content analysis.
Political Psychology 23(3), pp. 417437. DOI: 10.1111/0162-895X.00292
Hesse-Biber, S.N. and P. Leavy. 2006. The Practice of Qualitative Research. Thousand Oaks: Sage
Publications.
Holliday, A. 2007. Doing and Writing Qualitative Research. 2nd Ed. Thousand Oaks: Sage
Publications.
Hsieh, H. and S.E. Shannon. 2005. Three approaches to qualitative content analysis. Qualitative
Health Research 15, pp. 12771288.
158 Jared J. Wesley
King, G., R. Keohane and S. Verba. 1993. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton: Princeton University Press.
Krippendorff, K. 2004. Content Analysis: An Introduction to Its Methodology. 2nd Edition. Thousand Oaks: Sage Publications.
Laitin, D.D. 1986. Hegemony and Culture: Politics and Religious Change among the Yoruba.
Chicago: University of Chicago Press.
Laver, M. 2001. Why should we estimate the policy position of political actors? In M. Laver (ed.)
Estimating the Policy Positions of Political Actors. London: Routledge. pp. 13
Lewis, J. and J. Ritchie. 2006. Generalizing from qualitative research. In J. Ritchie and J. Lewis
(eds.) Qualitative Research Practice: A Guide for Social Science Students and Researchers.
Thousand Oaks: Sage Publications. pp. 263286.
Manheim, J.B., R.C. Rich and L. Willnat (eds). 2002. Empirical Political Analysis: Research Methods in Political Science. 5th Ed. Toronto: Longman.
Marshall, C. and G.B. Rossman. 1989. Designing Qualitative Research. Thousand Oaks: Sage.
Merriam, S.B. (ed.) 2002. Qualitative Research in Practice. San Francisco: Jossey-Bass.
Miles, M.B. and A.M. Huberman. 1994. Qualitative Data Analysis: A Sourcebook of New Methods. Thousand Oaks: Sage Publications.
Morse, J.M., M. Barrett, M. Mayan, K. Olson and J. Spiers. 2002. Verification strategies for
establishing reliability and validity in qualitative research. International Journal of Qualitative Methods 1(2), pp. 1322.
Morse, J.M. and L. Richards. 2002. Read Me First for a Users Guide to Qualitative Methods.
Neuman, W.L. and K. Robson. 2007. Basics of Social Research: Qualitative and Quantitative
Approaches. Canadian Edition. Toronto: Pearson.
Phillips, L. 1998. Hegemony and political discourse: The lasting impact of Thatcherism. Sociology 32(4), pp. 847867. DOI: 10.1177/0038038598032004011
Platt, J. 2006a. Evidence and proof in documentary research: Part I, some specific problems of
documentary research. In J. Scott (ed.) Documentary Research. Volume 1. Thousand Oaks:
Sage Publications. p. 83.
Platt, J. 2006b. Evidence and proof in documentary research: Part II, some shared problems of
documentary research. In Documentary Research, edited by J. Scott. Volume 1. Thousand
Oaks: Sage Publications. p. 105.
Punch, K.F. 2005. Introduction to Social Research: Quantitative and Qualitative Approaches. 2nd
Edition. Thousand Oaks: Sage Publications.
Putnam, R. 1993. Making Democracy Work: Civic Traditions in Modern Italy. Princeton: Princeton University Press.
Richerson, P.J. and R. Boyd. 2004. The Origin and Evolution of Cultures. London: Oxford University Press.
Shively, W. Ph. 1998. The Craft of Political Research. 4th Ed. Upper Saddle River, NJ: Prentice
Hall.
Smith, C.A. and K.B. Smith. 2000. A rhetorical perspective on the 1997 British party manifestos. Political Communication 17(4), pp. 457473. DOI: 10.1080/10584600050179068
Snape, D. and L. Spencer. 2006. The foundations of qualitative research. In J. Ritchie and J. Lewis
(eds.) Qualitative Research Practice: A Guide for Social Science Students and Researchers.
Tashakkori, A. and Ch. Teddlie (eds). 2003. Handbook of Mixed Methods in Social and Behavioural Research. Thousand Oaks: Sage Publications.
Weinroth, M. 2004. Rituals of rhetoric and nationhood: The liberal anti-deficit campaign, 1994
1998. Journal of Canadian Studies 38(2), pp. 4479.
Wesley, J.J. 2009. In search of brokerage and responsibility: Party politics in Manitoba. Canadian
Journal of Political Science 42(1), pp. 211236. DOI: 10.1017/S0008423909090088
Wesley, J.J. 2011a. Analyzing qualitative data. In Explorations: A Navigators Guide to Research
in Canadian Political Science, edited by K. Archer and L. Youngman-Berdahl. 2nd ed.
Toronto: Oxford University Press.
Wesley, J.J. 2011b. Code Politics: Campaigns and Cultures on the Canadian Prairies. Vancouver:
UBC Press.
Wesley, J.J. 2011c. Observing the political world: Quantitative and qualitative approaches In
K. Archer and L. Youngman-Berdahl (eds.) Explorations: A Navigators Guide to Research
in Canadian Political Science. 2nd Edition. Toronto: Oxford University Press.
part ii
From text to political positions

viadiscourseanalysis
Introduction
Veronika Koller
The project and workshop From Text to Political Positions, from which the present volume derives, aims at reconciling mostly computerassisted qualitative and
quantitative approaches to analysing the language used by political actors. As we
have seen in the previous section, automated quantitative analysis requires a rigorous qualitative motivation to validate the results and their possible interpretations
in their political context. The integrated text analyses that spring from such a
mixed-methods approach are in turn intended to promote understanding of how
political processes shape, and are shaped by, texts, and how stance and attitude is
conveyed by political actors. Insights into those phenomena are meant to provide
concrete applications in terms of, for example, opinion polling and voting advice.
Discourse analysts traditionally favour the qualitative manual analysis of
relatively small text samples such as extracts from speeches, newspaper articles
or blog entries. Advocates of critical approaches to discourse analysis combine
the description of such texts with addressing how language use and discursive
behaviour brings about or reinforces social power (Fairclough 1985). Although
analysts have not adopted a uniform definition of discourse, the term can be
regarded as referring to language use as social practice that is based on conceptual representations (cf. Fairclough 2003; van Dijk 2003). Such representations are theorised as abstract conceptual structures of knowledge, beliefs, values,
goals and the emotions around them, all of which lead to particular attitudes and
behaviours, including the production of stance in discourse. Each instance of discourse has the potential to activate a particular interpretive frame in which a text
or part thereof makes sense to the recipient. However, there is no deterministic
relationship between texts and frames, as activation of the latter depends just as
much on the conceptual representations held by recipients as well as the context
of reception, including the psychological disposition of the reader or listener.
Argumentation theory and its emphasis on rational deliberation, as frequently
found in political discourse analysis (see chapters by Montesano Montessori and
Eleveld), cannot be complete without looking at those contexts and taking into
account the affective component of conceptual representations that is at work
164 Veronika Koller
in them. Discourse producers in the political domain recognise the importance

of linking argument to emotions when they employ narratives or storylines. As
Johnstone (2001:635) puts it: The essence of humanness, long characterized as
the tendency to make sense of the world through rationality, has come increasingly to be described as the tendency to tell stories, to make sense of the world
through narrative.
In cognitive terms, narratives reflect conceptual representations that have
a temporal sequence, i.e., scripts, as well as values and goals. According to
Czarniawska and Gagliardi (2003), narratives fulfil a wide range of functions, the
most important for political discourse being:
projection of particular representations of reality (ideational function)
strengthening group identity
persuasion
projection of alternative worlds and, last but not least,
entertainment
To the extent that they are characterised by conventionalised ways of using language that are determined by the above, and other, communicative purposes, narratives can be regarded as a class of genres. As such they comprise news reports
and histories but also genres that are usually associated with fictional texts yet are
also utilised by political actors. An example of the latter is fairy tales, as Lakoff
(1992) has shown in his study of the conceptualisation and justification of the first
Iraq war in the discourse of the then U.S. government. Here, critical discourse
analysis (CDA), cognitive semantics and stylistics can be usefully combined to
account for persuasion in political discourse. Another step in that direction is Van
Leeuwens chapter (this volume) on the linguistic style of politicians.
Critical discourse analysts seek to uncover the links between texts, orders of
discourse and social contexts, to investigate their relation to the cognitive affordances contained in texts, which function as rhetorical, persuasive devices. The
approach is critical in that it seeks to uncover such implicit affordances and by
doing so to ultimately contribute to equality and social change. It is no surprise
then that much, if not most, critical analysis has been done on political discourse,
mediating between the analysis of actor-issue relations in politics and fine-grained
lexical-semantic analysis. While the first tends to understand discourse in very
broad and not necessarily linguistic terms, the latter often uses tools from computational linguistics to investigate, but also restrict itself to, patterns of word
use. CDA provides a missing link by positing that the micro-level of text is linked
to the macro-level of socio-political context indirectly, via the meso-level of discourse practice, i.e., the patterns of producing, distributing, receiving and adapting
instances of discourse.
Part II. Introduction 165
In its socio-cognitive version (see also below), CDA theorises that conceptual
representations are at work at all three levels, but are most pertinent at the levels of
discourse practice and social context, where they can help to interpret the describable linguistic features of texts: Representations cannot be read off texts, but we
can still ask how they, and the ideologies they form, are both reflected and shaped
by linguistic features. So, at the meso-level of discourse practice, we find procedural scripts of certain discursive practices as well as conceptualisations of discourse participants and genres to be of relevance. Conceptual representations and
the dominant ideologies they combine into are theorised to interpret the findings
from the text at the macro-level of social context. For the greater part, the chapters
in this section adopt frameworks developed as part of the CDA project. In doing
so, they complement the studies presented in the first section by providing sophisticated theories of the role of discourse in linking political structure and agency.
They engage in rigorous lexical-semantic analysis and carefully argued interpretive
analysis that connects characteristics of language use with aspects of the political
context. Consequently, the empirical approaches of these chapters provide new
insights into the complexity and dynamics of language use in the political structure in which they function. They take text analysis a level upwards to the political
implications of the findings, and possible applications (as discussed in Part III).
The chapters comprising the present section employ and advance a number
of theories underlying the methods for modelling political positions that were
presented in the previous section. Drawing on various strands within CDA as well
as on stylistics, especially text-world theory, the authors analyse various genres of
political discourse in the Mexican, Dutch, Northern Irish and U.S. contexts. More
specifically, they investigate how social and institutional change is effected, or
fails to be effected, and actor coalitions are forged through discourse (Montesano
Montessori, Eleveld), how changes in the political context impact on the language
use of political actors (Filardo-Llamas), what stylistic variations in politicians
language use give rise to popular perceptions of linguistic style (Van Leeuwen)
and what role new media play in recontextualising elements of political discourse
(Boyd). In all cases, the focus moves from content and linguistic e lements to discursive constructions of social realities.
The five papers in this section address a range of different political contexts, focusing on both adversarial and consensual (or communicative and coordinative, to use the terms adopted in Elevelds study) stances as expressed in
discourse. A further difference, which is related to the nature of stance, is the
time dimension; while the synchronic or quasi-synchronic1 studies contrast and
1. The data in Montesano Montessoris study were produced at different points in time but are
treated as a synchronic corpus.
166 Veronika Koller
c ompare stances and attitudes, the diachronic ones chart developments from
adversarial to consensual (Filardo-Llamas) or indeed from consensual to adversarial (Boyd) discourse. Interestingly, the data used to exemplify the varying stances
and their development are largely the same across the five studies, showing an
emphasis on speeches. This genre is perhaps the most typical one instantiated in
political discourse and, like the discourse as a whole, is characterised by verbal
oppositionality and stance taking.2 Within the data samples, however, there is
considerable variation, comprising inaugural and parliamentary speeches along
with state-of-the nation speeches and declarations. Following Reisigls (2008:246)
distinction between polity as the political frame for political actors, policy as the
content-related dimension of political action, and politics as political processes,
we would expect the speeches associated with political processes, i.e., parliamentary speeches, to realise adversarial discourse. The speeches by Northern Irish
politicians analysed by Filardo-Llamas and those concerning the immigration
debate in the Netherlands that Van Leeuwen investigates indeed contribute to an
adversarial discourse. However, polity speeches such as declarations, when delivered in reaction to similar ones from opposing actors (cf. chapter by Montesano
Montessori), can perform the same function, while Boyds study shows that policy
speeches such as inaugural addresses can bring about more adversarial reactions
than election speeches associated with politics. Broadening the perspective, the
genre is understood as a link in a chain of recontextualisation in Boyds study,
which investigates public reactions to different speeches and compares them
against the speeches themselves. Other data, as those analysed by Eleveld, consist
of policy documents and interviews with political actors, enabling the researcher
to triangulate the analysis.
Four of the five authors explicitly align themselves with the CDA project,
whereas Van Leeuwens study draws on stylistics. Within CDA, we can broadly
distinguish between a critical realist position ultimately inspired by neo-Marxist
thinking (e.g., Fairclough 1989, 2010), a socio-cognitive strand (e.g., Van Dijk
2003, 2008; also Koller 2005) and a discourse-historical approach (Reisigl and
Wodak 2009). In their linguistic take on post-positivist policy analysis, Montesano
Montessori and Eleveld both adopt the discourse-historical approach. The fact
that both studies work with synchronic or quasi-synchronic data (see n.1) corroborates the fact that despite its name, the discourse-historical approach can also
be used to investigate a discursive formation at a particular moment in time as
long as it is regarded as linked to prior and future discourses. Elevelds methodology is directly derived from this, comprising a qualitative manual analysis
2. An exception to the inherent oppositionality of speeches as a genre is the eulogy.
of nomination, predication and argumentation to investigate triangulated sets of

data and thereby trace the emergence of a discourse coalition around a particular
policy. Montesano Montessoris analysis of speeches is equally indebted to the
discourse-historical approach encompassing the three steps of content, argumentation and linguistic analysis. Corpus software to generate concordances is used to
identify semantic profiles of empty signifiers, such as modernisation and democracy. This is a good example of how the use of corpus-linguistic tools can and
indeed should involve a clearly defined qualitative element to account for quantitative corpus-linguistic results.
The socio-cognitive approach is taken, again, by Eleveld, and by FiliardoLlamas, where both draw on socio-cognitive notions in discourse analysis to
investigate frames and storylines as a means to create coherent discourse worlds
that communicate subjective political views. In general, we can state that the
behaviour of political actors, including their discursive behaviour is influenced,
and in turn influences, conceptual representations about themselves, others and
the world in general. Norms and values can be discursively disseminated and
enforced by discourse participants who, by dint of their social role within the
community, are in the position to produce and distribute more or less influential texts. Such texts are vehicles for their producers conceptual representations.
Other people receive these texts and doing so repeatedly under similar conditions of reception is likely to impact on their practices, their material reality and
indeed their own conceptualisations. In political discourse, this impact can be
seen as the producers main intention with the text, preferably to align recipients
representations with their own.
Eleveld employs such socio-cognitive notions to analyse and contrast storylines and their linguistic encodings, Eleveld infers interpretive frames such as
family life, diversity or human capital. While these encompass competing values
and conceptualisations, the communicated storyline or model of a certain socialsecurity system allowed actors to integrate aspects of the frame into their own
mental representations. Filardo-Llamas specifically draws on text-world theory and
discourse-space theory to trace the mental representations that are both conveyed
and evoked in the political discourse under investigation. To infer those conceptualisations, her qualitative manual analysis focuses on transitivity, including pronouns, and metaphor as the interface between discourse and cognition. Results
suggest that political actors address multiple audiences of narrowly and broadly
defined in-groups, and, as the political structure in Northern Ireland changes to
facilitate reconciliation, increasingly address out-group members as well.
Where Filardo-Llamas diachronic study traces a development from adversarial to consensual discourse, Boyds chapter on YouTube message-board comments on Obamas election, victory and inaugural speeches shows a change from
168 Veronika Koller
consensus to adversity. Comparing the online comments with the speeches they
relate to, the author engages in quantitative corpus analysis of key words which
is followed by a qualitative manual focus on pronouns and evaluative lexis. This
combined analysis allows for insights into how parts of the speeches are recontextualised and shows how the different sub-genres trigger different reactions from
different audiences.
Like Montesano Montessoris research on government vs. opposition speeches,
van Leeuwens paper also involves the analysis of adversarial discourse in speeches
by a populist Dutch politician and a former Dutch minister of integration. Starting
from public perceptions of the respective politicians style as clear or woolly, Van
Leeuwen applies a linguistic checklist to understand how those perceptions may
have come about. Like Boyd, he engages in a quantitative corpus analysis, here of
lexis, cohesion and syntax, which is complemented by a qualitative manual analysis of metaphor. Van Leeuwens results clearly show how the different parameters
accumulate to form a particular linguistic style that is intuitively perceived and
evaluated by the voting public.
The main contribution of the chapters in this section lies in their potential to
reconcile the different traditions in text analysis as summarised in Wesleys chapter (this volume). Thus, the qualitative manual analyses which allow Eleveld to
infer interpretive frames and Filardo-Llamas to identify text worlds complement
Dahlberg and Sahlgrens computational model of issue framing, while Boyd relies
on similar data and research questions as Gryc and Moilanens computational
sentiment analysis but enriches it with an integrated qualitative and quantitative
methodology that rests on a sophisticated linguistic foundation. Elevelds and
Filardo-Llamas socio-cognitive approach would lend itself very well to the use
of semi-automated semantic annotation (see e.g., Semino et al. 2009 on semantic
tagging to identify metaphor candidates, and Kleinnijenhuis and Van Atteveldts
relational model of semantic networks, this volume). Further integration of computational and discourse analytical ways in a layered analysis of contextual, textual
and linguistic charateristics requires a cross-disciplinary approach that involves
new challenges to proceed from texts to political positions, as discussed in the
following chapters.
References
Czarniawska, B. and P. Gagliardi (eds.). 2003. Narratives we Organize by. Amsterdam: Benjamins.
DOI: 10.1075/aios.11
Fairclough, N. 1985. Critical and descriptive goals in discourse analysis. Journal of Pragmatics
9(6), pp. 739763. DOI: 10.1016/0378-2166(85)90002-5
Fairclough, N. 1989. Language and Power. London: Longman.
Fairclough, N. 2003. Analysing Discourse: Textual Analysis for Social Research. London:
Routledge.
Fairclough, N. 2010. Critical Discourse Analysis. 2nd ed. London: Longman.
Johnstone, B. 2001. Discourse analysis and narrative. In D. Schiffrin, D. Tannen and H.E.
Hamilton (Eds), Handbook of Discourse Analysis. Oxford: Blackwell, pp. 635649.
Koller, V. 2005. Critical discourse analysis and social cognition: Evidence from business media
discourse. Discourse & Society 16(2), pp. 199224. DOI: 10.1177/0957926505049621
Lakoff, G. 1992. Metaphor and war: The metaphor system used to justify war in the gulf. In
M. Ptz (ed.) Thirty Years of Linguistic Evolution: Studies in Honour of Ren Dirven on the
Occasion of his Sixtieth Birthday. Amsterdam: Benjamins, pp. 463481.
Reisigl, M. 2008. Rhetoric of political speeches. In R. Wodak and V. Koller (eds), Handbook of
Communication in the Public Sphere. Berlin: de Gruyter, pp. 243269.
Reisigl, M. and R. Wodak. 2009. The discourse-historical approach. In R. Wodak and M. Meyer
(eds.), Methods of Critical Discourse Analysis. 2nd ed. London: Sage, pp. 87121.
Semino, E., A. Hardie, V. Koller and P. Rayson. 2009. A computer-assisted approach to the analysis of metaphor variation across genres. In J. Barnden, M. Lee, J. Littlemore, R. Moon,
G. Philip and A. Wallington (eds.), Corpus-Based Approaches to Figurative Language: A
Corpus Linguistics 2009 colloquium. Birmingham: University of Birmingham School of
Computer Science, pp. 145153.
van Dijk, T. 2003. The discourse-knowledge interface. In G. Weiss and R. Wodak (eds.), Critical Discourse Analysis: Theory and Interdisciplinarity. Basingstoke: Palgrave, pp. 85109.
van Dijk, T. 2008. Discourse and Context: A Sociocognitive Approach. Cambridge: Cambridge
University Press.
chapter 8
The potential of narrative strategies

inthediscursive construction of hegemonic
positions and social change
Nicolina Montesano Montessori
Utrecht University of Applied Sciences
This chapter illustrates ways in which narrative strategies contribute to the

construction of hegemonic processes of social change. Narratives have transformational power because they shape new imaginaries about social life, help to
legitimate them and create consensus. It is argued that effective hegemony constitutes power over social reality and will be generally accepted as a matter of
course. This chapter is based on a study of the struggle for hegemony between
former Mexican President Gortari and the Zapatista movement and integrates a
Gramscian view on hegemony with discourse theory (Laclau and Mouffe 1985).
The study contributes to the development of discourse theory on hegemony,
using the concept of three levels of abstraction, or orders of discourse, found in
Critical Discourse Analysis (Fairclough 2001, 2003); and a discourse-historical
approach (Reisigl and Wodak 2001). A comparison is made with Elevelds study
of storylines (this volume) and Filardo Llamas text worlds, leading to the presentation of criteria for qualitative research and some ideas for further research.
True hegemony requires a discourse that constitutes power over social reality by
establishing its own common sense. An essential part of political struggle is to
establish and maintain a discourse that catches on in an on-going negotiation
between groups. Hegemonic positions can be found in texts by analysing constructions of (alternative) imaginaries of social life. Such constructions are often
built around a narrative structure that functions to establish a text-intrinsic logic.
We therefore focus on narrative structures in political argumentation, as a persuasive strategy for hegemonic positioning. A discourse-analysis framework
emerged from a study of the discursive struggle between the former president of
Mexico, Salinas de Gortari (19881994), and the Zapatista liberation movement
(EZLN) (Montesano Montessori 2009), in which narrative plays an important role.
The EZLN broke into rebellion on the very day in which the North Atlantic Free
172 Nicolina Montesano Montessori
Trade Agreement (NAFTA) was implemented, on January 1, 1994. The architect

of this agreement was President Salinas de Gortari, a technocrat who had hoped
to launch Mexico into the first world and into modernity. Participating in NAFTA
implied a series of political and economic changes, including state and constitutional reform. However, the EZLN opposed this shift towards a global economy. It
reminded Mexico of its indigenous roots and claimed its famous slogan: Enough
is enough. No Mexico without us. The EZLN occupied several communities in
the state of Chiapas, located in the southeast of Mexico. Their struggle aimed
to achieve justice, democracy and liberty, to which the claim for autonomy for
indigenous communities was added at a later stage. After a short and violent conflict with the Mexican military, they changed their struggle into a discursive war.
They organized civil meetings, made ample use of the mass media to spread their
word and they organized international meetings against globalization and neoliberalism (cf. Montesano Montessori 2009, Chapters 1,2).
In this study, the struggle was recontextualised as a different narrative setting
of an initial local response to a global move of the government. The data selected
for the analysis included: the inaugural speech of Salinas de Gortari and his 1st,
3rd and 6th annual State of the Nation Reports presented to the Congress, and
the first four Declarations of the Lacandon Jungle presented by Subcomandante
Marcos, the leader of the EZLN (see Montesano Montessori 2009:155163). In
order to perform the analysis, a theoretical-analytical framework was needed in
order to identify hegemonic aspects of these discourses. The result was a framework which integrated a Gramscian view on hegemony with discourse theory
and two main directions in critical discourse analysis (CDA). Thus, the theory
of the dialectical-relational approach to CDA (Chouliaraki & Fairclough 1999;
Fairclough 2003, 2006) was used to identify whether narratives functioned at the
levels of myths or imaginaries, while a discourse-historical approach (Reisigl and
Wodak 2001) was employed to analyse the narratives. This chapter presents the
development of a research model tailored to this specific political context, with
emphasis on the role of narrative strategies (Montesano Montessori 2011).
Data as narratives
The underlying assumption for this study was that both the EZLN and Salinas
strategically hoped to gain hegemonic acceptance for their respective projects
concerning the future of the nation. It was also assumed that narrative strategies
would play a significant role in this struggle for hegemony, based on the fact that
in times of crisis, narratives play a significant role to formulate potential alternatives to the status quo (Jessop 2002). Gaining hegemony requires achieving
Chapter 8. Narrative strategies inthediscursive construction 173
consensus for a particular project, as will be discussed below. Narratives provide

a means to disseminate this envisioned project because narratives can be understood as stories with a beginning, a middle, and an end that contains a conclusion
or some experience of the storyteller. Telling a story is normally connected with
some unusual event and some complication in the course of the events depicted
(Titscher et al. 2000:215).
An important aspect of narrative is that it allows elements of information
to be put together in a logical order that creates a story of emplotment, that is
to say, a logical sequence of events (Somers 1994:616). As such, narratives play
a central role within political discourse, especially in times of disagreement and
struggle (Jessop 2002). Narrative provides the means to define and legitimate and/
or delegitimate a particular project. Narrative allows for the imagination of a different political space from which new identities can emerge as well as new links
between causes and effects. Patterson (2002:2) emphasises the strategic function
of narratives due to its potential to create, to make meaning, to maintain, to
resist, to change, to prove, to falsify, to argue, or to control something or someone.
She quotes Ricoeur (1991:33): What is sought is no longer an intention hidden
behind a text, but a world unfolded in front of it. The power of a text to open a
dimension of reality implies in principle a recourse against any given reality and
thereby a possibility of the critique of the real. Narrative helps to unfold and
legitimate a particular worldview (see Van Elfrinkhof, Maks and Kaal, this volume:
3.4). In view of our Mexican case study, the shift to a global economy undertaken
by Salinas marked, indeed, a moment of major restructuring by transforming the
state and the economy to the needs of the global market. Salinas used narratives
to defend and promote his project in ways which were contested by the EZLN. It
is possible, then, to recontextualise the data as hegemonic narratives (Montesano
Montessori 2009:144148).
A theory on hegemony
Gramsci (1971) views hegemony as a form of power which is based on consent
rather than on repression and in this view hegemony implies the general acceptance of a specific dominant doctrine. Furthermore, he inverts the classical Marxist
view, in which the basis determined the superstructure, and claims that culture
resides in a superstructure and is created there, and that intellectuals as well as
subordinate classes have a favourite position in creating a culture with underlying
ideologies. They have a capacity to create a new historical block, that is, to form
a new social force consisting of various social groups willing to compromise and
to overcome their own interests in order to form new alliances and to support
potential new projects. This view endorses a central role to language and grammar
to create a shared vision, to legitimate it and to find support for it. Furthermore,
it defines this mobilisation of civil society into a collective will as a war of position, as opposed to a war of manoeuvre that is physically fought with weapons.
Gramsci presents a new notion of power in which power ceases to be a thing to
be seized and becomes a matter of establishing new relations between powers in
society (Gramsci 1971; Howarth 2000). The emphasis placed on language puts
forward another argument to focus on narratives in this particular chapter and in
the wider context of this volume.
Discourse theory for a discourse-analytic framework

An analytical framework was designed for the analysis of Government vs.
Zapatista discourse, applying a critical approach in combing relevant methods to
this particular case (Montesano Montessori 2009, 2011). Discourse Theory (DT),
as developed by Laclau and Mouffe (1985), is a poststructuralist theory, which
rejects the structuralist view that language is a stable, unchangeable structure.
It abandons Saussures distinction between langue and parole, and the idea of
fixed relations between signs and signifiers. DT is based on a radical constructivist ontology that implies that discourse constitutes the social world by giving
it meaning. One of the central assumptions of DT is that social fields (society
at large, but also social units such as organisations and families) are constituted
through differences and conflicts. These differences can be dealt with according
to two types of logic: the logic of equivalence, in which one identity negates the
other, thus dividing social space into two antagonistic poles; and the logic of difference, which tends to reduce existing differences and to marginalize them in order
to share a common goal (Laclau and Mouffe 1985:12734; Howarth 2000:107).
Discourse Theory claims that processes of establishing meaning are political by
nature. In this sense, the relation between the universal and the particular, as
described in DT, is also relevant. The universal represents an empty space and
filling this space with meaning is a hegemonic operation. Various groups will
aim to formulate a meaning for this universal space. Whenever a group achieves
general support for their particular vision or project, the group has established
hegemony. This coincides with the Gramscian notion of the creation of a collective will (Torfing 1999:175). For example, in the case study on Mexico, the EZLN
attempted to construct a collective will in the direction of a project for a free, just
and democratic Mexico. Here, Mexico operates as a universal, empty space and
the EZLN attempts to fix its meaning through new concepts of regional autonomy,
democracy and nationalism (Montesano Montessori 2009:4854; 98101). It takes
political power to fix the meaning of a subject, that is, to articulate it (Laclau and
Mouffe 1985:105). Rearticulation, as will be seen later, entails the rewriting of an
existing formulation of social reality.
DT distinguishes a series of linguistic concepts which are central to the process of gaining hegemony and each have their own particular role to sustain intrinsic logic in a narrative. Four of these hegemonic concepts are of particular interest
in our study: empty signifier, nodal point, myth and social imaginary.
1. Empty signifiers are signifiers without a particular signified. Due to the
absence of a directly observable relation to a particular signified, they represent a
totality that cannot be fully captured or obtained from the words. For example, the
concepts of nationalism and democracy are empty signifiers that obtain meaning
only by charging them with contextual meaning. The act of giving them meaning
is a hegemonic operation (Laclau 1996/2007:44). In my view, empty signifiers are
relevant in the creation of a hegemonic position for several reasons. First, due to
their lack of meaning they have the potential to unite different groups behind a
shared common cause. Each group can fill in the meaning in their own way. In the
case of the EZLN, for instance, the movement succeeded at least in its rhetoric
to unite civil society and the indigenous people behind its empty signifier of a
free, just and democratic Mexico. As such, empty signifiers play a significant role
in the logic of difference, due to their potential to unite different groups behind
shared goals. When successful, this procedure may lead to a new hegemonic bloc,
which may then gain hegemony (Laclau 2005:131). Filardo Llamas (this volume)
gives such an example of a political shift towards the logic of difference in the
speeches by Paisley and Adams in Northern Ireland (2006). And Eleveld (this
volume) shows how the life course perspective represents an empty signifier.
2. Nodal points help to create and sustain the identity of a certain discourse by
constructing a web of fixed meanings and thus to stabilise a particular discourse.
Nodal points can be defined as privileged signifiers that fix the meaning of a signifying chain (Laclau and Mouffe 1985:112). Salient examples are the free market in
Thatcherite political discourse, blood pressure in traditional medicine, and a pressure point in acupuncture (Howarth 2000; Phillips and Jrgensen 2002). Salinas de
Gortaris modernisation may have served as a nodal point in his discourse.
3. Myth is a new space of representation (Howarth 2004:261) that sets boundaries to what Laclau calls the social imaginary (Laclau 1990). The imaginary of
an alternative to the hegemonic structure is constructed by those elements that
are experienced as being absent from the existing dominant structure (Laclau
1990:63). Thus, the EZLNs perspective on a free, just, democratic Mexico could
well be considered a myth: an alternative to what, in their eyes, was a country in
the hands of a corrupt and illegitimate government that submitted the country to
the interests of international capitalism.
4. Once a myth absorbs all social demands and all possible dislocations, it crystallises in a social imaginary (Laclau 1990) that allows perspectives for change.
A myth becomes a social imaginary once all social groups or at least the majority can project their demands into that imaginary and consider it a solution to
their needs (Montesano Montessori 2009:9498; 2011:172).
Key concepts of discourse theory in relation to narrative strategies

These four concepts provide useful clues for the analysis of constructions of
hegemonic positions. Taken together, these concepts allow analysing the formulation of a myth, the formation of alliances and the rearticulation of social reality.
Hegemony is constructed through the creation of alliances (the formation of a new
historical bloc) by defining shared meaning (Montesano Montessori 2009:96). I
claim that it is in this very sense that narrative can play a significant role, since it
allows the agent to construct a story with mythical implications and which contains strategies of legitimation so as to unite different groups. Empty signifiers support this process. This perspective seems to be supported by Eleveld (this volume)
when she refers to unlikely discourse coalitions.
However, Laclau and Mouffes version of discourse theory also poses a series
of problems. It is developed at a high level of abstraction that makes it impossible
to directly apply these concepts to the analysis of texts. Furthermore, the four
key concepts remain under-theorised, so that it is impossible to identify nodal
points or empty signifiers in concrete data. For instance, throughout the analysis,
modernisation was identified as a key signifier in the narrative of Salinas, and
democracy in the narrative of the EZLN. But it was impossible to tell whether
these should be considered empty signifiers or nodal points. This very problem
and its relation to the concepts of imaginary and myth, has been raised in existing
literature on DT (Howarth 2004; Norval 2000). A more specific theory was needed
concerning the relationship between discourse and social processes (structure and
agency) in order to understand whether the data operated as myth or as social
imaginary. It is at this point that it seemed particularly productive to integrate
Discourse Theory with Faircloughs theoretical approach (2003), Chouliaraki and
Fairclough (1999) and the methodology of the discourse-historical approach in
CDA, (Reisigl and Wodak 2001). In the final stage of the analysis, after the data
collection and identification of relevant discourse elements, a particular tool for
computer- assisted language analyisis was used, WordSmith, to help us identify
these elements in their linguistic context (cf. Montesano Montessori 2009, 2011).
Three levels of abstraction in relation to discourse theory

In order to structure the research frame, we have applied the theoretical framework of the dialectical-relational approach to CDA (cf. Montesano Montessori
2009, 2011 for further details) to the analysis of the narratives. Faircloughs
(2003:2328) model of three levels of abstraction, includes social structures, social
events and social practices. Their linguistic and discursive counterparts are respectively language, texts and orders of discourse.
Social structures are the most abstract, and refer to the enduring and general characteristics of societies, such as an economic structure or kinship. Its
linguistic counterpart is the language system shared by a social group (e.g.,
Spanish, in this case, or Japanese).
Social events are concrete events in social life, e.g., a visit to the doctor. The
linguistic counterparts are written, spoken or visualised texts.
In between are the social practices that mediate between structures and events.
They regulate what is generally done in particular areas of social life. This tends
to be regulated by institutions (such as education) or organizations (a school)
(Fairclough 2003:2325). The discursive counterpart is an order of discourse,
which is a socially structured articulation of discursive practices (including
both genres and discourses) which constitutes the discursive facet of the social
order of a social field, such as politics, media or education. (Chouliaraki and
Fairclough 1999:114). Social structures provide a set of possibilities, but the
relationship between what is structurally possible and what actually happens,
between structures and events, is a very complex one (Fairclough 2003: 23).
This tension is played out in social practice. At this level of abstraction, the
discursive counterpart is formed by orders of discourse which operate in
three main ways, namely as genres (ways of acting), discourses (ways of representing) and styles (ways of being). Orders of discourse are closely related
to power because they involve certain procedures that constrain what can be
said by whom and on what occasions (Foucault 1984 [1982]: 109; Fairclough
1992:51). Orders of discourse empower and constrain the social agent at
the same time. Therefore, orders of discourse represent the tension between
structure and agency. Structures press their influence on institutional practice,
which is considered an order of discourse in CDA, but it is within these orders
of discourse that agents can take up their social struggle.
Applying the three levels of abstraction to the analysis of hegemonic constructs

In the case of the Government vs. Zapatista movement in Mexico, the Mexican
Constitution could well be considered a social structure articulated in the Spanish
language. The President sought to modify several articles of the Constitution,
while the EZLN called on the Constitution to confirm that national sovereignty
lies in the hands of the Mexican people rather than in the State (Montesano
Montessori 2009:189). The social events are, in this case, the pronunciation of
the annual State of the Nation reports by the President to the Congress, and the
pronunciation of the declarations of the EZLN to the Mexican nation at large.
Social practices include the political institutions of the executive power and the
Congress and the constitutional obligation for the President to pronounce annual
national reports to the Congress. In the case of the EZLN, the procedure was
decided upon by the leadership, to pronounce declarations on a regular basis to
inform the nation at large, as well as the international community, about the status quo and the intentions of their struggle. However, the opposing parties each
belong to a different order of discourse. The Presidents speeches belong to the
order of discourse of the federal government, while the declarations of the EZLN
belong to the order of discourse of a social movement. The subjects of our analysis
are the discourses that represent these points of view (rather than on genres or
identities). This implies, that, the analysis given here focuses on ways in which
Salinas and the EZLN represented the situation in Mexico. The fact that Salinas
and the EZLN each belong to a different order of discourse, implies that each of
them is endowed with different sources and degrees of power, and restrictions to
this power. For instance, Salinas was empowered to fulfil the role of President of
Mexico. As such he could make amendments, give orders for military operations,
decide to seek a peaceful solution to the EZLN conflict, and so on. He was also
restricted in that he needed support from Congress to execute certain decisions.
The empowered leader of the Zapatista movement was Subcomandante Marcos.
In his capacity, he was entitled to decide on a strategy for political change (armed
struggle or political representation), and mobilise and train new members of the
movement. But he could not give orders to the federal army or make amendments
to the Constitution (Montesano Montessori 2009:147).
The order-of-discourse approach also facilitates a new perspective on hegemony. The model suggests that social reality is a layered account of structures which
provide possibilities and restrictions, and agents that can or cannot modify these
structures. The events, whatever actually happens, occur at the most concrete level.
In my view, we can suggest that hegemony requires having an impact on at least
the level of social practice/order of discourse, and/or at the higher level of social
structures. Thus, the discourse-theoretical concept of creating a historic bloc
and charging empty signifiers with meaning seems an important initial step to
establish hegemony, but it will only be achieved once this shared meaning has a
significant impact on the status quo. Hence, for the EZLN to change the democratic structure in Mexico, it is not sufficient to redefine the concept in terms of
indigenous beliefs about leadership and democracy. It is also necessary to have
this new vision implemented through the modification of power structures. In
other words, narrative alone is not enough: it will only construct hegemony if it
contributes to the modification or recontextualisation of social reality at the level
of social practice.
Narrative strategies and the discourse-historical approach

Let us now move from theory to methodology and the analysis of data. A discourse-historical analysis operates on three levels:
1. The establishment of the topics of a specific discourse;
2. Analysis of discursive strategies (including strategies of argumentation);
3. The linguistic means (as types) and the specific context-dependent linguistic
realisations (as tokens).
(Reisigl and Wodak 2001)
The research procedure entails a dialectical process in which the researcher goes
back and forth between theory, data and methods through operationalisation, verification and explanation. Wodak et al. (1999:3335) distinguish a series of macro
strategies, e.g., in the construction of national identity. These include strategies
of dismantling, justification, transformation and construction that are supported
by micro strategies of singularisation, delegitimisation, as well as argumentative
strategies. I suggest that narratives may serve as an encompassing strategy which
can be instrumental in transforming or dismantling existing projects and which,
simultaneously, contain a series of microstrategies. In the case of social change, I
would add the strategy of imagination, which serves to depict an alternative to the
status quo, which is a myth in discourse theoretical terms. Strategies of cooperation may serve to construct a historical bloc that is required to perform the actual
change at the level of social practice. Since the issue of quality criteria in CDA was
discussed during the T2PP conference, I would like to spell out the quality criteria
formulated for this research:
1. Transparency in terms of the epistemological, ontological, methodological
orientation and organisation of a research project, as well as in respect of the
position of the researcher.
2. Meaningful patterns (quantified or not) in the discursive and textual analysis

should emerge, be shown and explained.
3. The interaction between theory, methodology and analysis of data should lead
to a result that is more than the sum of the parts. Theory and methodology
may help to analyse the data, but this analysis in itself will provide information
about the theories and methodologies used, so that new perspectives may be
opened up.
4. These dynamics should lead to new insights regarding the social problem that
is being addressed.
5. The outcome of the research should have evident relevance for social agents
and institutions and so on.
(Montesano Montessori 2009:144,293296)
The meticulous analysis, the continuous dialogue between theory, methodology
and data analysis, the triangulation of theories and methods, including the use of
computer-assisted tools for linguistic analysis, provide sufficient quality to this
study to make up for not including purely quantitative analysis. In the light of the
related chapters in Part II (Eleveld and Filardo Llamas), it needs to be stated that
in post-positivistic social science, several different but related approaches combine: CDA, discourse theory, post-positivist policy analysis and text-world theory.
This is an enrichment rather than a problem, but it does require one additional
criterion for quality: the researcher needs to be transparent about choices made
to construct a research frame and the relevance of this framework to the data and
goals of the study. Every concept that one uses must be (re-)defined to manifest
the way in which it is operationalised.
Data analysis
Integrating CDA and discourse theory at the analytical level
The analytical procedures of the discourse-historical approach in CDA, make
it possible to relate discourse theoretical concepts to empirical data. When performing the content analysis two saliently distinct projects for the Mexican state
clearly emerged (see Montesano Montessori 2009:208209 and 2011 for details).
The President presented an ambitious project of state reform and modernisation,
which would include state reform, the establishment of and the participation in
NAFTA, the development of a program for solidarity and reform of parts of the
constitution. On the other hand, the EZLN proclaimed the removal and abolition
of the government, and the creation of political space for civil society to formulate
a new agenda for Mexicos national future. Drawing on the further elaboration that
hegemony requires an impact at the level of orders of discourse, the content analysis suggested that the narrative of the President, could be considered an imaginary
in the sense that his project, indeed, had an impact on social practice and orders of
discourse. However, the project of the EZLN, remained a myth and failed to make
any substantial changes in Mexicos national political arena. Its attempts to create
a historic bloc remained at the level of social events and text and failed to have
an impact at the level of orders of discourse.
Analysing discursive strategy

The second stage of the discourse-historical method involves the analysis of
discursive strategies by means of the analysis of the argumentative structure.
Following Van Eemeren and Grootendorst (1992), arguments were analysed in
terms of premises and the main arguments used to defend the two parties own
projects and, in the case of the EZLN, to deligitimise the projects of the opponent.
Following Wodak et al. (1999) the analysis of discursive strategies entailed the
analysis of strategies of self-legitimation, dismantling undesired realities, transformation of these realities as well as strategies of constructing new and desired
realities (Montesano Montessori 2009:217244). This analytical procedure
revealed the relation to the discourse theoretical concepts of articulation and rearticulation of social reality. The analysis revealed that the President envisioned a
republican nationalism (one nation, one people) and a liberal democracy, while
the EZLN struggled for ethnic nationalism and a radical democracy, which entails
the deepening and expansion of the democratic process (Montesano Montessori
2009:5152; Olesen 2005). In other words, both parties rearticulated the future of
Mexico in two different directions. At this point of the analysis, modernisation
was identified as a key concept in the narrative of the President, and democracy
in that of the EZLN. However, it remained impossible to resolve whether these
were empty signifiers of nodal points.
The identification of empty signifiers and nodal points

through linguistic analysis
Having raised the question concerning the identification of empty signifiers and
nodal points in data (Montesano Montessori 2009, 2011), the analysis made it
possible to identify two key concepts: modernization in the case of Salinas de
Gortari and democratization in the narrative of the EZLN. These two concepts
were submitted to a meticulous study of its co-text in the third and final stage of
the analysis. This analysis was performed by making use of the corpus linguistic
tool WordSmith. This is a rather unusual way of using WordSmith, which is mostly
used to perform corpus-based research on very large quantities of data for the
analysis of underlying quantitative syntactic-semantic differences (Scott & Tribble
2006). In this study, WordSmith was used for the analysis of the concordance of
these two concepts in a relatively small data set. Nevertheless, it turned out to be a
very adequate instrument for a close analysis of keywords in their syntactic environment. A sample of the output is provided in Figure 1 (for a more substantial
report on this study, see Montesano Montessori 2009:264271; and 2011).
N
1
2
3
4
5
6
7
8
9
10
11
Concordance
reso. Avanzamos ahora hacia el cambio. La
modernizacin
r la descentralizacin del sector, junto con la modernizacin
zar la politica, la economia y la sociedad. La
modernizacin
tros desafios. Dar impulso y movimianto a la modernizacin
yor parti..cipatcin de la sociedad, y sera una modernizacin
ionalista, democrtica y popular. Ser una
modernizacin
sociales que ya ocurrieron, y pera lograr la
modernizacin
eblo. Nuestro camino para el cambio ser la
modernizacin
I pueblo. El Estado sera rector efectivo de la
modernizacin
dispensable su participacin para orientar la modernizacin
atriotas. Alentar y conducir el esfuerzo de
modernizacin
ce Mxico es indispensable p
de su infraestructura. Manten
de Mxico es tambin inevita
a la que por voluntad y neces
popular porque tendr un clar
nacionalista porque reafirma I
y el cambio por la via instituci
nacionalista, democratic y
de Mxico, pero sta solo ser
en condiciones de justicia y
nacional que respetar nue
Figure 1. Keyword in context: Modernizacin in the inaugural speech (IS)

of Salinas de Gortari.
The keyword modernisation appeared as an urgent and inevitable project in the

inaugural speech, and as a process that is well on its way. Verbs contain a clear
indication that progress had been made and that modernisation had already transformed the country. It was claimed to have had a major impact on fundamental
sectors of the economy. Modernisation appears as an urgent and inevitable project in the inaugural speech, and as a process that is well on its way. In SNR1
it appears as a chosen strategy that affects various sectors of the economy and
segments of the population. It requires a series of new attitudes. This is reflected
mainly in the verbal system, which indicated material processes referring to processes of doing (Halliday 1994:109). Examples in the inaugural speech (Figure1)
included dar impulso, to trigger (4) lograr, to achieve (7). In SNR3 it is presented
as being implemented in most sectors by the 3rd year of Salinas term, and as having reformed the country by the 6th year (SNR6). In the Sixth State of the Union
Report, most of the occurrences are embedded in processes of transformation,
which are the result of modernisation. It is grammatically supported by a verbal
system that indicates material achievement and progress. Examples are that modernisation was advanced (promover) in the mining sector and it has transformed
the garbage sector. Progress has been made: modernisation has transformed the
country and it has had a major impact on fundamental sectors of the economy. It
was therefore safe to assume that modernisation represented a nodal point in the
narrative of Salinas around which action should be taken.
The EZLN narrative (Figure 2) represented Democracy as a moment situated in the future. The EZLN called on civil society to support its struggle for
a democratic system. The most frequent use of the words surrounding democracy indicate transition to, e.g., with verbs like to achieve, to struggle, to search.
In a Hallidayan sense, these are creative types of material processes (Halliday
1994:111). Because the goal lies in the future it can only be referred to as an
uncertainty, accounting for the frequent use of the future or subjunctive tense.
Democracy was therefore identified as an empty signifier in its purest sense. It
defines a lack, but it does not have a signified because it is not yet existent. It is
only present as a projection, a desire to struggle for.
N
1
2
3
4
5
6
7
8
9
10
11
12
otra las anim et afn de lucha por la

NOS: La paz vendr de la mano de la
n la bsqueda del trnsito pacific a la
NIVELES Y EN TODA PARTES, por la
a cuyo destello ilumine el camino de la
eun gobiemo nacional de transicin a la
o, principal obstculo para el trnsito a la
cimos: La Patria vive! Y es nuestra!
sectores del pais, de que el trnsito a la
ucacin, justicia, independencia, libertad,
unico que tenemos, la vida, para exigir
nto como gobierno de transicin a la
Concordance
democracia,
democracia,
democracia.
democracia,
democracia.
democracia
democracia
democracia!
democracia
democracia
democracia,
democracia
la libertad y la justicia para t

la libertad y la justicia para
A travs de la Convencin
la libertad y la justicia, a t
El caso chiapaneco solo
con las siguientes character
en nuestro pals. En la seg
Libertad! Justicia! D
era posible por la via elect
ypaz. El proceso prelecto
libertad y justicia para todos
al que se doten por si mis
Set tag word no.

329
2.622
593
1.985
969
2.482
362
2.714
533
509
1.751
2.397
Figure 2. Keyword in context: Democracia in the Third Declaration of the EZLN.
The results of this analysis, when related to the three levels of abstraction distinguished in CDA, provide evidence that the narrative of Salinas does indeed
operate as an imaginary (as opposed to the myth of the EZLN). The project of
modernisation in the narrative of Salinas represents an imaginary in that it did
modify the state, as well as the Constitution. At the discursive level the research
provided evidence that, within this imaginary, modernisation operates as a nodal
point. In contrast, in the case of the EZLN the result remains mostly at the verbal/
textual level. While their narrative has raised a series of ideas about the nature
of democracy, it has failed to recontextualise in any significant way the political
and public social practices that are connected with it at the national level. I therefore maintain that the EZLNs narrative, at least with respect to democracy, has
remained a myth. These findings are summarised in Table 1.
Table 1. A summary of the main research results on three levels of analysis.

Discourse-historical
approach (HDA)
Discourse Theory
Data analysis
Content analysis
Myth, Narrative
Narrative of the EZLN perceived

as myth, that of the President as
imaginary.
Analysis of discursive
and argumentative
strategies
Articulation, rearticulation
The EZLN rearticulates Mexicos

future in terms of an ethnic
nationalism and a radical
democracy; President Salinas de
Gortari as a republican nationalism
and a liberal democracy.
Analysis of linguistic
features
Identification of empty
signifiers and nodal points
democracy in the narrative of the

EZLN operates as an empty signifier;
modernisation in the narrative of
the government as a nodal point.
The role of narratives in relation to the discursive construction

ofhegemonic positions
The case study on Mexico allowed for a clarification of the potential relation
between narratives and orders of discourse. Orders of discourse represent a
system of power. They entail procedures about what can be said and done, the
distribution of different forms of capital, etc. It is because of these characteristics, that an order of discourse becomes the target of power struggle. From this
point of view, it is now possible to consider an order of discourse as a domain
or even a target of social struggle, and narrative as one of the main tools within
that struggle. Narrative, then, is a vivid tool and instrument in the struggle to
change an order of discourse, an institution, the status quo of a nation, or that of
a political system and the position of agents within them (Montesano Montessori
2009:148150).
The chapters by Eleveld and Filardo Llamas (this volume) show some similarities to the narrative approach sketched in this chapter. Eleveld explicitly pays
attention to narrative in policy constructions, which may reflect meaningful realities, function as argumentative strategies and to generate motivation for social
change. She clearly supports the view that narrative strategies contribute to processes of social and policy change. She adopts discourse theory when showing
in detail how the empty signifier life course perspective is being charged with
meaning by five parties that construct five different narratives on a shared story
line. Filardo Llamas case for text world, and especially ideological text world
analysis has close relations with the narrative approach. Text worlds provide the
ontology in which narrative linearity makes sense in chains of reasoning (cf.
Kleinnijenhuis and Van Atteveldt, this volume). Both construe dynamic networks between actors, objectives and actions. Narrative is a powerful instrument
in the articulation of positions (see also the section on narrative affordances of
Worldviews in Van Elfrinkhof et al., this volume, Section 3.4). Although these
studies concern different situations, they are each related to difference or conflict
in which narrative structure plays a powerful role in bridging between ideas and
actions. It is in this light that I would like to formulate some research questions
for further research:
1. What is the relation between ideological text worlds (based on text world
theory, Werth 1999) and narratives as described in this chapter? Are they
synonymous? Does one encompass the other? Can these theoretical concepts
complement each other?
2. Can a detailed text-oriented discourse analysis help to further develop postpositivist policy? If so, what criteria for success should be established? Is it
possible or necessary to quantify these criteria?
3. While the 2006 speeches of Adams and Paisley (Filardo Llamas, this volume)
in Northern Ireland seem to shift toward the logic of difference, an analysis of
empty signifiers in these and the 1988 speeches could lead to a better understanding of the discursive representation of the conflict from a discoursehistoric point of view.
Conclusion
This chapter outlines a theoretical-methodological framework designed for an indepth analysis of a specific political discourse in the light of a hegemonic struggle.
The framework integrates a Gramscian account on hegemony, key concepts from
discourse theory and theoretical and methodological notions of Critical Discourse
Analysis and is complemented by linguistic concordance analysis in order to identify empty signifiers and nodal points in empirical data. It has contextualised the
data as narratives and it has provided evidence to suggest that the two competing
narratives in the Mexican context operate at two different levels of abstraction.
It has presented narrative as a significant strategy in the analysis of discursive
constructions of hegemonic political positions. Narrative is a powerful tool in the
construction of hegemony through its potential to formulate and disseminate new
imaginaries as well as through its potential to achieve consensus for the political
project it envisions. However, narrative affordances are culture dependent and
therefore require adapted analytic approaches tailored to the context in which they
function. As this chapter demonstrates, a critical discourse approach, including a
historic approach, is considered powerful to fulfil this task by virtue of its layered
design and the variety of strategies it offers to select tools for the identification of
hegemonic strategies in a particular context.
Note
For a complete version of the outcome of this analysis, see Montesano Montessori (2009:303
306); and Montesano Montessori, Schuman and De Lange (2012).
References
Chouliaraki, L. and N. Fairclough. 1999. Discourse in Late Modernity. Rethinking Critical Discourse Analysis. Edinburgh: Edinburgh University Press.
Fairclough, N. 1992. Discourse and Social Change, Cambridge: Polity Press.
Fairclough, N. 2001. CDA as a method in social scientific research. In: R. Wodak and M. Meyer
(eds.), Methods of Critical Discourse Analysis, London: Sage, pp. 121-139.
Fairclough, N. 2003. Analysing Discourse. Textual Analysis for Social Research. London:
Routledge.
Fairclough, N. 2006. Language and Globalization. Abingdon/New York: Routledge.
Foucault, M. 1984 [1982]. The order of discourse. In M. Shapiro (ed.), Language and Politics,
Oxford: Basil Blackwell, pp. 108-138.
Gramsci, A. 1971. Selections from the Prison Notebooks. New York: International Publishers.
Halliday, M.A.K. 1994. Introduction to Functional Grammar, 2nd ed. London: Edward Arnold.
Howarth, D. 2000. Discourse. Ballmoor: Open University.
Howarth, D. 2004. Hegemony, subjectivity, democracy. In S. Critchley, O. Marchart, (Eds),
Laclau. A Critical Reader. Oxoford: Routledge, pp. 256276.
Jessop, B. 2002. The Future of the Capitalist State. Cambridge: Polity Press.
Laclau, E. 1990. New Reflections on the Revolution of our Time. London, New York: Verso.
Laclau, E. 1996/2007. Emancipation(s). London: Verso.
Laclau, E 2005. On Populist Reason. London: Verso.
Laclau, E. and C. Mouffe. 1985. Hegemony and Socialist Strategy. Towards a Radical Democratic
Politics, London: Verso.
Montesano Montessori, N. 2009. A Discursive Analysis of a Struggle for Hegemony in Mexico. The
Zapatista Movement versus President Salinas de Gortari. Saarbrcken: VDM.
Montesano Montessori, N. 2011. The design of a theoretical, methodological, analytical framework to analyse hegemony in discourse. Critical Discourse Studies 8(3), pp. 169-182. DOI:
10.1080/17405904.2011.586221
Montesano Montessori, N., H. Schuman and R. De Lange. 2012. Kritische Discoursanalyse. De
macht en kracht van taal en tekst. Brussels: Academic and Scientific Publishers.
Norval, A.J. 2000. Trajectories of future research in discourse Theory. In D. Howarth, A.J.
Norval and Y. Stavrakakis (eds.), Discourse Theory and Political Analysis: Identities, Hegemonies and Social Change. Manchester: Manchester University Press, pp. 219223.
Olesen, T. 2005. International Zapatismo. The Construction of Solidarity in the Age Of Globalization. London: Zed Books.
Patterson, W. (Ed.). 2002. Strategic Narrative: New perspectives on personal and cultural narratives, Lexington, M.A.: Lexington Books.
Phillips, L., and M.W. Jrgensen, 2002. Discourse Analysis as Theory and Method. London: Sage.
Reisigl, M., and R. Wodak. 2001. Discourse and Discrimination. Rhetorics of racism and antiSemitism. London: Routledge.
Ricoeur, P. 1991. Life in quest of narrative. In D. Wood (Ed.), On Paul Ricoeur: Narrative and
Interpretation, London: Routledge, pp. 2033.
Somers, M.R. 1994. The narrative constitution of identity: A relational and network approach.
Theory and Society 23, pp. 605649. DOI: 10.1007/BF00992905
Titscher, S., M. Meyer, R. Wodak and E. Vetter. 2000. Methods of Text and Discourse Analysis.
London: Sage.
Torfing, J. 1999. New Theories on Discourse. Laclau, Mouffe and iek. Oxford: Blackwell
Publishers.
Van Eemeren, F.H. and R. Grootendorst, 1992. Argumentation, Communication, and Fallacies.
A Pragma-Dialectical Perspective. Hillsdale, NJ: Lawrence Erlbaum.
Werth, P. 1999. Text Worlds: Representing Conceptual Space in Discourse, London: Longman.
Wodak, R., R. de Cillia, M Reisigl, and K. Liebhart. 1999. The Discursive Construction of National
Identity. Edinburgh: Edinburgh University Press.
chapter 9
Christians, feminists, liberals, socialists,

workers and employers
The emergence of an unusual discourse coalition
Anja Eleveld
University of Leiden
How can actors reach an agreement on the future of social policy when they
interpret the world around them very differently? Around the turn of the century diverse groups in The Netherlands, such as workers, employers, feminists,
and Socialist, Christian and Liberal political parties, reached an agreement
on the introduction of a life-course perspective in the social-security system.
The system was to anticipate the ways individuals prefer to distribute their
time between work and other activities like care, education, leisure, etc. This
chapter explains how this unusual coalition between parties and stakeholders
could emerge, with attention to methodological aspects concerning the study
of discourse coalitions. The argument is that a central storyline in a life-course
perspective works as a cohesive, uniting different actors, because each actor can
interpret the signifier life-course perspective in his/her own way.
1. Introduction
Just after the turn of this century, the issue of a life-course based system of social
security arose on the Dutch policy agenda. The social-security system was to be
reformed into a life-course based system. This idea was shared by diverse groups,
such as supporters of family policy; labor unions; employer unions; social democrat, Christian and liberal political parties and feminists. This roughly meant that
they agreed on a new social security system that should anticipate on the ways
individuals prefer to distribute their time between work and other activities like
care, education, leisure, etc. In 2006, the consensus on the life-course perspective
was followed by the introduction of the Life-Course Arrangement, a new instrument of social security law. How can the realization of this unusual coalition be
explained? This chapter seeks to provide an answer to this question using concepts
which are developed in post-positivist policy analysis. It additionally examines
190 Anja Eleveld
how post-positivist research on policy change can benefit from hermeneutics and
methods developed in Discourse Historical Analysis.
New institutionalist approaches to policy analysis often experience difficulties explaining institutional changes, particularly with respect to social security
policies (Hemerijck and Visser 1997; Pierson 2001; Powell and DiMaggio 1991).
Therefore, new institutionalist theory increasingly turns to post-positivist policy analysis (Schmidt 2008). Scholars working within the field of post-positivist
policy analysis particularly recognize the central role of values in the process of
policymaking (Fischer 2003; Hajer and Wagenaar 2003). For such scholars, problems are not a given in the world waiting for policymakers to take action; instead,
these problems are actively constructed in the form of narratives or storylines.
According to Hajer (1995), an important function of a storyline is that it can
both simplify the various aspects of a complex problem and facilitate different
groups to read their own narrative in this storyline. As a result, a storyline not
only suggests unity in complex situations, it is also able to create unity among
different groups. In fact, the main property of a story line is that it sounds right.
In case a number of actors in the context of a set of practices use a particular set
of storylines a discourse coalition may emerge that can facilitate change. Drawing
on the concepts developed by Hajer, this study poses the hypothesis that a broad
discourse coalition could emerge because different actors were able to read their
own narrative into an appealing storyline on the life-course perspective. On this
assumption, the study seeks to contribute to a further development of research
methods on the formation of discourse coalitions within texts, an approach that
has so far remained rather undeveloped.
To understand the concept of discourse coalition within the Dutch political context this study uses Schmidts (2002, 2008) distinction between the coordinative and the communicative discourse. For Schmidt, the construction of a
discourse coalition takes place at the level of the coordinative discourse, which is
located in the policy sphere and consists of individuals and groups, such as civil
servants, elected officials, experts and organized interest and activists, who are
involved in the creation, elaboration, and justification of policy and programmatic ideas (Schmidt 2008:310). The coordinative discourse can be contrasted
with the communicative discourse in that it operates in the political sphere in
which various individuals and groups seek to convince the public. Whereas in
countries with a single authority, such as France and the UK, the communicative discourse dominates, the coordinative discourse is particularly important in
corporatist and multi-party countries such as The Netherlands where close consultations between different (small) parties are imperative in the decision-making
process. In these multi-party countries the communicative discourse is made subordinate to coordinative discourse, because it could easily disrupt carefully construed compromises. A discourse coalition is, then, understood to be a coalition
Chapter 9. The emergence of an unusual discourse coalition 191
between different discursive positions within the coordinative discourse which

aims to influence the policy agenda.1
This study on the formation of discourse coalitions distinguishes analytically
between different functions of narratives in the policy process. To enhance the
potential revisability of the study (cf. Wesley, this volume), hermeneutic methods,
which are commonly used in post-positivist policy analysis, are complemented
with tools derived from Discourse Historical Analysis. Analysis of the results leads
to reflections on the implications of this approach for a post-positivist policy analysis of discourse coalitions and considerations as to what extent other methods
that are presented in this volume can contribute to the analysis.
2. Methodology: A post-positivist analysis of narratives
Post-positivist policy analysis is a term for a broad range of analytical approaches
in the field of policy analysis, which seeks to move beyond an objectivist conception of reality. Scholars working within this field primarily object to the positivist
claim that policy problems can be approached in a technocratic, pure empiricist way. Rather, policy problems are constructed in the form of narratives. For
analytical purposes this section distinguishes between the following overlapping
functions of policy narratives: they can reflect meaningful realities (narrative as
meaningful practice); they can function as an argumentative strategy (narrative as
argumentative practice); and they can generate social change (narrative as transformative practice).
Story telling is considered an important instrument to frame problems,
because the elements of a specific story become related with a particular way of
looking at these elements. An analysis of the narratives, expressed in different
interpretive communities eventually reveals the underlying interpretive frames
through which different meanings are expressed (Schn and Rein 1977, 1994;
Yanow 2000). A policy frame selects and highlights certain features of a policy
problem while it ignores other features. In addition, a policy frame connects the
salient features into a pattern that is coherent and easy to grasp (Schn and Rein
1977). Instead of identifying, the right interpretation, the analyst should conceive
conflicts between interpretive frames as an opening for a new understanding of
the policy problem (Yanow 2000) or to reframe the policy problem (Schn and
Rein 1994). The analysis of narratives as a reflection of meaningful realities thus
enables us to comprehend how a discourse coalition is endorsed in different interpretive frames.
1. Note that this definition of a discourse coalition diverges from Hajers definition which also
takes the practices into account in which storylines are uttered.
192 Anja Eleveld
Although Schn and Rein (1994) argue that (policy) narratives can reveal the
tacit frames that underlie the problem settings, particularly in their later work,
they do not strictly distinguish between narratives and frames. This study follows
Brandwein (2006) who proposes to separate baseline categories of thought from
their interpretive products (see also Linder 1995). According to Brandwein, an
interpretive frame consists of base line categories of thought which can give rise
to different narratives. Together, these elements form the interpretive frame in
which certain aspects of a policy problem is highlighted while other features are
ignored. In case the baseline categories of thought are analytically separated from
their interpretive products the process of interpretation comes to the fore.
In interpretive policy analysis, the analysis of frames requires first of all an articulation of the policy narratives that are presented in a policy text. Consequently,
the analyst has to find out which baseline categories of thought give rise to these
narratives. For these purposes the analyst has to engage in a hermeneutic analysis
of (spoken) texts. This means that after a first interpretation of the text or interview,
further analysis is required in order to check if the interpretation is (in-)complete
or erroneous. In this process the analyst eventually interprets the texts in the context of other related texts, which may also be her own intent and thoughts. In fact,
the analysts may go back and forth from text to context several times. This hermeneutic circle ends when further analysis does not lead to any new interpretation
(Schwarz-Shea 2006; Yanow 2006).
In post-positivist analysis, narratives are not only considered as meaningful
interpretations of reality, they are also deployed as argumentative strategies of
policymakers (Fischer 2003; Fischer and Forrester 1993). Fischer, for example,
argues that policymaking becomes like story telling. That is, finding or reconstructing the appropriate storyline is a central form of agency for the political
actor (Fischer 2003:88). For this reason he claims, that political conflicts are often
due to disagreement about the basic storyline, or a competition between narratives
and counter-narratives. In addition, a policy narrative can appear in the quality of
a narrative of decline or control (Stone 1988). In a narrative of decline attention
is being put on the way things are getting worse and worse. In a narrative of control, while it is admitted that things are going bad, possibilities of interference are
advanced. In this struggle on the basic storyline, policymakers can purposefully
deploy these narratives.
Finally, narratives or storylines can affect the emergence of discourse coalitions and generate policy or even institutional change. In this respect Hajer challenges Sabatier (1987) who argues that policy change can be attributed to the
formation of advocacy coalitions. For Hajer, in contrast, Sabatier fails to acknowledge that individual beliefs may be altered, because of the emergence of new
central storylines (Hajer 1995). Hajer thus explicitly views narratives as transformative practices that are capable of changing discursive positions.
3. The Dutch policy context

The Dutch political system can be characterized as consensual, that is, the power
over political decisions is distributed among different political party coalitions.
From the Second World War on, the Christian democrats (CDA) were almost
continuously present in governing coalitions. However, between 1994 and 2002,
when the ideas on life-course policy emerged from the coordinative discourse,
The Netherlands were governed by the purple coalition. This coalition consisted
of the other two important political parties the social democrats (PvdA) and
the right-wing liberals (VVD) and the relative small left wing liberals (D66),
leaving the Christian democrats in the opposition for years. Another feature of
the Dutch system is that it is corporatist. Corporatism can be described as an
empirical relationship between interest groups and the government that is based
on exchange (influence for support), and on cooperation rather then competition
(Andeweg and Irwin 2002:39, cited by Kuipers 2004:51). In a corporatist society,
the relationship between government and interest groups is based on exchange.
Dutch society can be characterized as corporatist as governments cooperate with
the labor unions and employer unions. The most influential labor unions are the
social-democratic FNV and the Christian CNV, of which the FNV is by far the
largest. The largest employers organization is VNO-NCW.
4. The selection of the data and the methods of analysis
The hypothesis that a broad discourse coalition could emerge, because different
actors were able to read their own narrative into an appealing storyline on the
life-course perspective, led to the expectation that the vagueness of a central storyline of the life-course perspective facilitated the inclusion of various interpretive frames. Hence, the first step in the analysis was to find the central storyline.
Second, the purpose was to figure out how the signifier life-course perspective
was filled with meaning in different interpretive frames and related narratives. In
a third step, the analysis shifted to the way actors assigned different meanings to
the life-course perspective.
With respect to the first step, the three most influential documents referring to
the life-course perspective were selected in order to construct its central storyline.2
In the second step, a number of actors were interviewed about the way they interpreted the signifier life-course perspective and whom they thought contributed
2. CDA 2001; SER 2001; Ministry of Social Affairs and Employment 2002.
194 Anja Eleveld
significantly to the meaning of the signifier and the introduction of the life-course
perspective in the coordinative discourse.3 On the basis of those interviews, five
key actors were selected who participated in the coordinative discourse.4 In an
attempt to capture the baseline thought categories, the first two steps were inspired
by the hermeneutic circle. This means that the researcher went back and forth
to the interviews, the most important writings of the interviewees, other related
documents and her own thoughts. However, whereas hermeneutic processes tend
to be less transparent than other more quantifiable methods, it was decided to
repeat the analyses deploying a method that could more easily be replicated by
other researchers.
This second method builds on Discourse Historical Analysis (DHA) developed by Wodak and others (Reisigl and Wodak 2009; Wodak et al. 1999).5 DHA
entails a detailed study of texts. Each claim that is made in a text needs to be studied and categorized. DHA formulates a number of heuristic questions for this kind
of text analysis, of which the following three are relevant for this study:
1. How are persons, objects, phenomena/events, processes and actions named
and referred to linguistically (strategies of nomination)?
2. What characteristics, qualities and features are attributed to social actors,
objects, phenomena/events and processes (strategies of predication)?
3. What arguments are employed in the discourse in question?

(Reisigl and Wodak 2009:93)
This third heuristic question is further elaborated in a study on the construction of
national identities in which Wodak et al. (1999) distinguish between strategies of
justification, strategies of construction, transformative strategies and strategies of
3. For this purpose the following actors were interviewed: Bovenberg (expert in SER, advisory
council of the government which also consists of social partners and advisors of the CDA party,
Bijleveld (FNV); Cuyvers (member of the family council and advisor of CDA); Driessen (FNV);
Dolsma (VNO-NCW); Evenhuis (Member of the emancipation council until 1997 and of its
successor Tecena until 2000; advisor of emancipation policy); De Geus (former CDA Minister
on Social Affairs and Employment); Kastelein (CNV); Goudswaard (crown-appointed member
of SER); Leijnse (crown-appointed member of SER); Schippers (Member of the emancipation
council until 1997 and of its successor Tecena until 2000; advisor of emancipation policy);
Slootweg (CNV); Van der Braak (VNO-NCW); Wierda (public official emancipation affairs).
4. These actors were: Bovenberg, Cuyvers, Evenhuis, Leynse and Schippers (for details see
former reference).
5. Montesano-Montessori put the idea forward to apply DHA during a conversation we had
in June 2010 in Amsterdam (see also Montesano Montessori 2009).
dismantling. These categories were adapted to the structure of the analyzed texts.6
Deploying DHA, all claims made in selected texts originating from the key actors
were categorized and analyzed according to the heuristic questions.7
A few additional notes need to be made with respect to the construction of
the interpretive frames. First, the frames presented in Table 1 cannot be attributed
to the individual selected key actors, as they are also entangled in webs of meaning. Second, the content of the publications of these key actors are richer than
the interpretive frames seem to suggest. In addition, the frames were not jointly
exhaustive of all possible constructions or necessarily the most salient. The purpose of these constructions, then, was not to categorize all possibilities, but to analyze the discursive position of the actors involved in the coordinative discourse.
The third purpose of this research involved the analysis of the discursive position of political parties, social partners, policymakers and interest groups who
referred to the central storyline. After a quick scan, the following actors were
selected: CDA, PvdA and D66 (political parties); FNV, CNV and VNO-NCW
(social partners); the family Council and insurance companies (interest groups);8
and emancipation policymakers. Consequently, a number of articles, reports
and pamphlets were picked out for further analysis.9 To identify the interpretive
6. The claims in the text were categorized in one of the following classes: social trends, problematization of social trends, undesired reality considering these problematizations, persons and
entities contributing to these undesired realities, desired reality.
7. After having read their most important publications that referred to the life course perspective (publications that also already informed the hermeneutic process), a selection was made in
regard to those publications that were most representative for their ideas during the period that
the life course perspective became popular: Bovenberg 2001, 2003; Cuyvers 1996; Evenhuis
1999a, 1999b; Leijnse 2001a, 2001b; Schippers 2001.
8. Insurance companies were primarily involved in the technical execution of the Life Course
Arrangement. Their role was less prominent in the generation of the life course perspective
discourse; therefore, this paper leaves the insurance companies out.
9. The analyst first gathered as much as possible documents originating from diverse actors that
seemed to refer to the life course perspective. As the success of the storyline on the life course
perspective reached its peak in 2001/2002, the focus was put on document that appeared before
2002. The most important/influential documents were selected for further analysis: Balkenende
(CDA) 2002; Van der Braak (VNO-NCW) 2001; CDA 1997, 2000, 2001a, 2001b, D66 2001,
2002; Dolsma (VNO-NCW) 2004; CNV 1997; Emancipatieraad (advisory council emancipation policy) 1996; FNV 1993, 2001a, 2001b, 2003a, 2003b; Kastelein (CNV) 1997; Kastelein and
Van Suijdam (CNV) 2001; De Ley (VNO-NCW) 2001; Ministry of social affairs and employment 2001; PvdA 1996, 2002a, 2002b; Terpstra and Oostveen (CNV) 2001; Kamerstukken II
(Parliamentary documents emancipation policy) 20002001, 27 061, no. 3, 19992000, 27 061,
no. 1-2; Westerlaken (CNV) 1996.
196 Anja Eleveld
frames that were used in these documents, a list of key signifiers was selected
that could be attached to each separate interpretive frame.10 After a signifier had
been traced down, the meaning of the signifier in the specific context determined
whether this signifier was related to one of the identified interpretive frames. In
this way it was possible to determine how different actors assigned meaning to the
life-course perspective.
5. Results of the analysis
For scholars engaged in hermeneutic research, it may be no surprise that the more
transparent DHA did not render salient results. On the other hand, DHA could
provide more precise information regarding the argumentative structure of the
storylines/narratives and the baseline categories of thought. The results are summarized here.
5.1
The story on the life-course perspective
The following central storyline could be constructed:

Different trends in modern society such as the increased labor market participation of women, individualization, ageing and globalization have caused major
transformations in individual life courses. In regard to these latter trends, changed
life courses are also imperative. The institutions of social security are, however,
insufficiently tailored to these demands and the (required) modern life courses,
particularly concerning those stages of life in which workers have to combine
different activities such as work, caring and education. In addition, people desire
more balance and say in the way they distribute their time between these activities. This mismatch between, trends, desires and the system of social security can
only be solved in case social policy is reconciled with modern life courses.
10. For each interpretive frame different key signifiers were identified. Family life: income of a
family, the costs of children, social and human capital of children, taking care for the children
by yourself; Human capital: human capital, economic costs, employability, education, life long
learning; New risks: new risks, manufactured risks, internal risks, external risks, distribution
of responsibilities, saving for social security; Emancipation: (distribution of) paid work and
care, labor participation of women, transitions of men and women, individualization of social
security arrangements, human capital, economic independence of women. Taking care of the
children by yourselves; Diversity: diversity, multiversity, quality of life, choice options, leisure,
sabbatical, individualization of social security.
The structure of the central storyline shows how the storyline functions as an
argumentative strategy. In the first place, the storyline offers a solution for current and expected problems. Second, the problems are constructed in a way that
stresses the gravity of the problems. For example, the deployment of nominalizations, like globalization, ageing and individualization, make these problems look
inevitable. The storyline further appeals to some contradictory feelings. Whereas
on the one hand, the storyline expresses fear of staying behind in the international
economic competition, on the other hand, the storyline provokes anxious feelings
about the loss of nationally cherished values, such as taking care of the care for
the children by ourselves and an enhanced quality of life. At first glance, the
storyline thus includes a narrative of control that is multi-interpretable.
5.2
One storyline, five interpretive frames
On the basis of the hermeneutic analysis and the Discourse Historical Analysis,
the researcher constructed five interpretive frames (Table 1): family life, diversity, emancipation, human capital and new risks. Each interpretive frame distinguishes between baseline categories of thought (appeals and warrants; the
constructions of society and human being) and their interpretive products (problematization; solution; the meaning of the life-course perspective). It should
be noted that the construction of the baseline thought category should not be
understood as an essentialist fixed idea, but as a contingent comparison between
positions.
Table 1 demonstrates that the interpretive frames appeal to very different values, such as traditional family values; quality of life; equality between men and
women; efficiency; and individual responsibility (fourth column). The dominant
constructions of society and human being are also diverse. Though they picture
in all instances human being as an autonomous chooser, some frames construct
human being either as a person whose preferences are primarily determined by the
broader social context or as a person whose preferences are primarily determined
by rational individual life plans. In addition corresponding with the construction
of human being, most frames tend to construct either a caring or an enterprise
society. The baseline category of thoughts particularly reveals some contradictions
between on the one hand family life and diversity, and on the other hand human
capital and new risks. The interpretive frame emancipation seems to occupy a
middle position in this respect.
Based on these different baseline categories of thought, each interpretive
frame constructs its own narrative of control that can be viewed as a meaningful
interpretation of the world as well as an argumentative strategy (second column).
198 Anja Eleveld
Table 1. Five interpretive frames.

Interpretive
frame
Narrative
Family life
Parents are not (financially) capable of caring for their children: a family
gap is emerging. Therefore a redistribution should take place of the peak
load in time and money during the life course
Diversity
Institutions of social security are based on the standard male life course,
whereas life courses (particularly those of women) and desires have
become more diverse. These institutions should anticipate on the diverse
range of activities that people (whish to) perform within one period of life
Emancipation
Institutions of social security and labor market policy are insufficient tailored to the increased labor market participation of women. Social security
should be based on transitional labor markets that facilitate leave arrangements for both men and women
Human capital
Current system of social security stimulates the depreciation of human capital.

Human capital is the key to the desired labor participation. A system of social
security that is based in investments in human capital would depart from a
better spread of education, work, care and spare time over a life course
New risks
Although the consciousness has grown that to a great extent people can
influence risks, the system of social security is still based on the idea of
external risks. In regard of the growing importance of manufactured risks,
the system of social security should be increasingly based on investments
incentives and individual responsibility
In addition, DHA reveals systematically that each narrative or frame constructs

its own scapegoat as it points at a distinct target group or entity that impedes the
realization of the right social system, or worse, profits from the current system,
such as elite groups (family life), the elderly (human capital), the welfare recipient (new risks), men (emancipation), and the dominant discourse (diversity).
Table 1 further shows that the meanings of the signifier life-course perspective are embedded in the narratives. Each interpretive frame thus fills the signifier
life-course perspective with different meanings (third column): (1) it means that
during the stage of family life, the income of households decreases (family life);
(2) it celebrates the increasing diverse life courses of women (diversity); (3)it
problematizes the distribution of paid work and care activities between women
and men (emancipation); (4) it refers to the investments and depreciation of
human capital (human capital); and (5) it refers to the way life courses increasingly become the product of ones own choices (new risks).
Meaning of the life-course

perspective
Appeals
and warrants
Dominant constructions of society

and human being
Perspective to view the

income gap during the stage
of family life
(traditional)
family values
A caring society
Perspective to view (and

celebrate) the diverse life
courses of women
Quality of life;
Freedom
A caring society
Perspective to analyze the

transitions of men and
women between the spheres
of work and care
Equality between A society that is both caring and

men and women enterprising
The autonomous chooser whose individual

preferences are primarily determined by
the broader social context
the broader social context

preferences are partly determined by to the
broader social context and partly related by
rational individual life plans.
Perspective to view the

Efficiency
investments and depreciation
of human capital
An enterprise society
Perspective that enables one

to view the life course as an
individual product in which
the individual makes his/her
own risk based decisions
A fair enterprise society
Optimal
combination of
efficiency and
justice; Individual
responsibility
The autonomous chooser whose

rational individual life plans
The autonomous chooser whose
rational individual life plans
DHA further reveals in detail other contradictions between the different

frames. For example, an analysis of predication strategies shows how the interpretive frame of family life challenges groups, such as the elite, social liberals
and the emancipation movement: groups that refer to the same central storyline.
DHA also discloses some incompatibilities between the frames diversity and
emancipation. The first frame openly attacks the emancipation model (and indirectly also the frames human capital and new risks), because in this model the
availability of women for paid work remains dominant.
Finally, with regard to the transformative function of narratives, DHA also
provides some additional results. This analysis particularly shows that signifiers
such as life-courses and life-patterns acquire a central place in claims concerning the construction of desired and undesired reality. In all frames those words
seem to be constitutive for the new discourse, giving rise to a changed perspective
on reality.
200 Anja Eleveld
5.3
Actors and frames
Table 2 demonstrates that most actors (political parties, social partners, interest
group and policymakers) who endorsed the central storyline draw on one or two
dominant interpretive frames. The actors involved thus assign different (dominant) meanings to the life-course perspective. In addition, in case those actors
address more than one dominant frame, the baseline categories of thought seem to
be compatible. On the other hand, it is remarkable that almost all actors somehow
refer to at least three or more interpretive frames. This means that the involved
actors seem to reconcile different values, and different constructions of the society
and human being. Still, fundamental differences between the actors did not disappear as most actors explicitly reject one interpretive frame (see fourth column).
Table 2. Actors and interpretive frames.
Actor drawing
on central
storyline
Dominant interpretive
frames
Important
interpretive
frames
Contested
frames
CDA
Family life
Human capital, Diversity Emancipation
PvdA
Emancipation, a combination
of Human capital and
New risks
Diversity, Family life
D66
Diversity
Emancipation, Human
capital, Family life
Policymakers
emancipation
(19952001)
Emancipation, Diversity
Human capital, Family

life, New risk
VNO-NCW
Human capital and New risks
Diversity
FNV
Emancipation and Diversity
Human capital
CNV
Family life
Human capital, Diversity Emancipation
Family Council
Family life
New risks
New risks
Diversity
6. An explanation on the emergence of a discourse coalition

The analysis of the data suggests that the unusual discourse coalition could emerge
because an attractive central storyline reflected the actors (policy) views, desires
and assumptions. The analysis that leads to this conclusion proceeds in the following steps. First of all, the study reveals that the storyline on the life-course
perspective contains an appealing narrative of control (narrative as argumentative
practice). Secondly, the signifier life-course perspective proves to carry different
meanings. In the coordinative discourse these meanings are embedded in five, to
some extent conflicting, interpretive frames (narrative as meaningful practice),

containing diverse baseline categories of thought that give rise to different narratives of control (narrative as meaningful and argumentative practice). Thirdly, most
actors who endorsed the central storyline deploy one or two dominant frames,
which lead us to the conclusion that the central storyline seems to have facilitated
the formation of a discourse coalition between actors who hold diverging beliefs.
Fourthly, the results show that actors themselves also refer to (partly) incompatible frames. This suggests that part of the success of the storyline can be
ascribed to the fact that it addressed win-win situations, which enabled the actors
to incorporate unfamiliar frames into their own body of thought: on the one hand,
advertizing a more relaxed life; on the other hand, promoting an increase in labor
participation; on the one hand encouraging the (collective) facilitation of more
time and money to take care of the children; on the other hand holding individuals responsible for care tasks, which are now defined as manufactured risks. The
central storyline thus facilitated the interchange of different interpretive frames
(narrative as argumentative and transformative practice). Importantly, the discourse coalition not only united different groups, a discourse historical analysis
of a number of key-texts also suggests that this coalition transformed the coordinative discourse, as reality was now viewed from the perspective of life-courses
and life-patterns (narrative as transformative practice).
In sum, the findings seem to support the hypothesis that the discourse coalition could flourish because different actors (political parties, social partners,
interest group and policy makers) were enabled to read their own narrative in
the storyline. The central storyline functioned as a cohesive glue uniting and mixing discursive positions in an attractive way that opened up possibilities for new
agreements between diverse actors whose perspective on reality had altered.
7. Implications
This case study shows how a post-positivist policy analysis of discourse coalitions
contributes to an explanation of the agenda setting process. An (analytical) distinction between the functions of the (policy) narrative proves to be useful and
can be recommended for future research on discourse coalitions.
The study further demonstrates that the Discourse Historical Analysis
provides similar results as the more subjective hermeneutic analysis. On the
other hand, DHA enables the analyst to be more precise and provides additional
information with respect to the research on discursive change. Still, conclusions
regarding changed discursive positions must be backed up by further research
on the discursive position of the actors involved in the period before the signifier
202 Anja Eleveld
life-course perspective was introduced. In addition, notwithstanding the fact

that the rise of a discourse coalition may be an important condition for institutional change, it should be acknowledged that the formation of a discourse
coalition does not necessarily result in institutional change. As a matter of fact,
the life-course perspective discourse coalition failed to bring about major institutional changes. That is, the life-course arrangement that was introduced in
2006 remained a rather trivial arrangement in the social security system, despite
proposals for an enlargement of this arrangement. Possibly, the success of the
storyline on the life-course perspective also explains the failure of major institutional change: concrete proposals with regard to a life-course based social
security revealed important differences of opinion between the involved actors.
Recall, in this respect, that a number of actors contested one interpretive frame.
There are also indications that agreement on the level of the coordinative discourse was not translated into communicative discourses. Yet, these hypotheses
are also in need of further investigation.
The question can be raised if quantitative methods can further add to the
analysis of discourse coalitions. For example, Dahlberg and Sahlgren (this volume)
show how quantitative methods can be useful for the investigation of issue framing. In regard to the study of discourse coalitions, this method can be interesting
to sort out how and when words such as life courses, life patterns, human capital, diversity, family gap entered the discourse; how these words are interrelated
with each other (within each frame) and how they constitute the discourse of
social policy in a new way. In addition, this method seems particularly useful for
the selection of texts and a preliminary analysis of the discursive positions of the
actors. On the other hand, these methods fail to provide a more substantive picture
of interpretive frames (baseline categories of thought, mutual contradictions and
exclusions, etc.). Also, with respect to the argumentative structure of policy narratives, qualitative contextual analysis of political and policy texts remain necessary.
A different issue concerns the question to what extent discourse theory based
on Laclau and Mouffe (see Montesano-Montessori, this volume) can contribute to
the study of discourse coalitions. For example, it could be argued that the signifier
life-course perspective transformed to an empty signifier and thus contributed
to the rise of a discourse coalition (compare Griggs and Howarth 2006). In my
opinion, the concept empty signifier is particularly useful in the study of communicative discourses, as it explains how, on a macro level of analysis, the union
of diverse groups (the population) is sustained by their common identification
with a new (empty) signifier (for example democracy or justice). Concerning
the analysis of the coordinative discourse, however, I believe that the concepts of
storyline and discourse coalition are more adequate. Second, it seems that the
articulation of an empty signifier is particularly helpful for the explanation of
political processes in a field that consists of antagonistic forces. In this case study,
however, it was not possible to identify oppositional forces. The discourse coalition could rather be characterized as inclusive: it attempted to address as many
identities as possible.
In conclusion, hermeneutic research and DHA are suitable methods for
research on discourse coalitions. Future research could profit from a mix of quantitative approaches such as proposed by Dahlberg and Sahlgren with a wider
deployment of discourse analytical methods. In case a discourse coalition emerges
in opposition to some other group(s), concepts and methods derived from discourse theory might also be considered.
References
Andeweg, R.B. and G.A. Irwin. 2002. Governance and Politics of the Netherlands. Basingstoke:
Palgrave.
Balkenende, J.P. 2002. Anders en Beter. Soesterberg: Uitgeverij Aspekt.
Bovenberg, A.L. 2001. Een belaste levensloop. ESB 2001 86, p. 171.
Bovenberg, A.L. 2003. Levensloop en sparen: Investeren in nieuwe sociale zekerheid. Sociale
Wetenschappen 46(4), pp. 6385.
Brandwein, P. 2006. Studying the careers of knowledge claims: applying science studies to legal
studies. In D. Yanow and P. Schwartz-Shea (eds.), Interpretation and Method: Empirical
Research Methods and the Interpretive Turn. M.E. London, New York: Sharpe, pp. 228243.
CDA 1997. De Verzwegen Keuze van Nederland. Naar een Christendemocratisch Familie- en
Gezinsbeleid. Den Haag: Het Wetenschappelijk Instituut voor het CDA.
CDA 2000. Discussienotitie ruimte voor elkaar.
CDA 2001a. De Druk van de Ketel. Den Haag: Wetenschappelijk Instituut voor het CDA.
CDA 2001b. Verkiezingsprogramma 20022006.
CNV 1996. Wie zorgt er voor de kinderen? Projectgroep kindvriendelijk beleid.
Cuyvers, P. 1996. Het gezin, sociale zekerheid en sociaaldemocratische onzekerheid. Socialisme
& Democratie 57(4), pp. 205219.
D66 2001. Tijd van leven. Werkgroep Levensloop.
D66 2002. Toekomst in eigen Hand. Verkiezingsprogramma 20022006.
De, Ley, R. 2001. Spitsuur in de Levensloop: Tijd voor Verscheidenheid. CDV 22 (7,8,9),
pp. 99104.
Dolsma, G. 2004. Zelf je leven lopen. ESB 89, p. 28.
Emancipatieraad. 1996. Met Zorg naar Nieuwe Zekerheid. Advies over een Gemancipeerd Inkomens- en Sociale Zekerheidsbeleid. Den Haag: Ministerie van Sociale Zaken.
Evenhuis, C.H.S. 1999a. Levensloop in de 21e eeuw (v/m): Introductie en discussiebijdrage.
In Levensloop in de 21e eeuw (v/m). Report of a Symposium, October 13, 1999, pp. 5-12.
Evenhuis, C.H.S. 1999b. Diversiteit in levensloop: Vrouwen als achterlopers of als pioniers. In
Levensloop in de 21e eeuw (v/m). ). Report of a Symposium the Oktober 13, 1999, pp. 1924.
Fischer, F. 2003. Reframing Public Policy. Oxford: University Press. 10.1093/019924264X.001.0001
204 Anja Eleveld
Fischer, F. and J. Forrester. 1993. Editors introduction. In F. Fischer, and J. Forrester (eds.), The
Argumentative Turn in Policy Analysis and Planning. Durham and London: Duke University Press.
FNV 1993. Veelkleurige vooruitzichten voor een samenleving, waarin mensen zelf kiezen en
samen delen, de Beleidsvisie van de FNV voor de komende jaren. FNV Convention 1993.
FNV 2001a. De visie van de FNV Vrouwenbond op Zorg, Arbeid en Inkomen aan het begin van
de 21ste Eeuw. Amsterdam: FNV Vrouwenbond.
FNV 2001b. Stop de tijd. Pamphlet.
FNV 2003a. Van prepensioen naar levensloop. Het FNV Alternatief. Discussion Paper. Amsterdam: Stichting FNV Pers.
FNV 2003b. Levens-loop-banen. Stichting FNV Pers.
Griggs, S. and D. Howarth. 2006. Metaphor, catachresis and equivalence: The Rhetoric of freedom to fly in the struggle over aviation policy in the United Kingdom. Policy and Society
25(2), pp. 2346. DOI: 10.1016/S1449-4035(06)70073-X
Hajer, M.A. 1995. The Politics of Environmental Discourse. Ecological Modernization and the
Policy Process. Oxford: Clarendon Press.
Hajer, M.A. and H. Wagenaar (eds.). 2003. Deliberative Policy Analysis. Understanding Governance in the Network Society. Cambridge: University Press. 10.1017/CBO9780511490934
Kastelein, M. 1997. Kinderen en sociale zekerheid. CDV 18 (12), pp. 637640.
Kastelein, M. and D. Van Suijdam. 2001. Het Arbeidsmarktbeleid van het CNV: Toetreden en
volhouden. CDV 22(7/8/9), pp. 184189.
Kuipers, S.L. 2004. Cast in Concrete? The Institutional Dynamics of Belgian and Dutch Social
Policy Reform. Delft: Eburon Academic Publishers.
Leijnse, F. 2001a. Het nieuwe werken. In P. Winsemius, L. van Driel, F. Leijnse and K. Vuursteen
(Eds), Naar een Nieuwe Maatschap. Den Haag: Ministerie van Economische Zaken,
pp. 59104.
Leijnse, F. 2001b. De Gevarieerde Verzorgingsstaat. Rede Uitgesproken bij de Aanvaarding van
het Ambt van Bijzonder Hoogleraar Sociaal-politieke Aspecten van de Verzorgingsstaat en
Overlegeconomie. Amsterdam: Vossius Pers.
Linder, S. 1995. Contending discourses in the electric and magnetic fields controversy. Policy
Sciences (28), pp. 209230.
Ministry of Social Affairs and Employment. 2002. Verkenning Levensloop, Den Haag: Ministerie
van Sociale Zaken en Werkgelegenheid.
Montesano Montessori, N. 2009. A Discursive Analysis of a Struggle for Hegemony in Mexico. The
Zapatista Movement versus President Salinas de Gortari. Saarbrcken: VDM.
Pierson, P. 2000. Increasing returns, path dependency and the study of politics. American Political Science Review 94(2), 251267. DOI: 10.2307/2586011
Powell, W.W. and P. DiMaggio. 1991. The New Institutionalism in Organizational Analysis.
Chicago: University of Chicago Pres.
PvdA 1996. Sociale Zekerheid bij de Tijd. Op weg naar het congres van 14 en 15 februari 1997.
PvdA 2002a. Samen voor de toekomst. Verkiezingsprogramma 2002-2006.
PvdA 2002b. Naar een nieuwe levensloop: voorstellen voor eigentijdse combinaties van arbeid,
zorg, scholing en Pensioen.
Reisigl, M. and R. Wodak, 2009. The Discourse-Historical Approach (DHA). In Methods of
Critical Discourse Analysis. London: Sage, pp. 87121.
Schippers, J.J. 2001. Arbeidsmarkt- en Emancipatiebeleid: De Vraag naar Diversiteit. Rede in

verkorte vorm uitgesproken ter gelegenheid van de aanvaarding van de leerstoel A
rbeidsen Emancipatie-economie aan de Universiteit Utrecht. Utrecht, 2001.
Sabatier, P.A. 1988. An advocacy coalition framework of policy change and the role of policyoriented learning therein. Policy Sciences 21, pp. 12968. DOI: 10.1007/BF00136406
Schmidt, Vivian A. 2002. The Futures of European Capitalism. Oxford: Oxford University Press.
DOI: 10.1093/0199253684.001.0001
Schmidt, Vivian A. 2008. Discursive institutionalism: The explanatory power of ideas and discourse. Annual Review of Political Science 11, pp. 30226. DOI: 10.1146/annurev.polisci.
11.060606.135342
Schn, D. and M. Rein 1977. Problem setting in policy discourse. In C.H. Weiss (ed.), Using
Social Research in Public Policy Making. Toronto: Lexington Books.
Schn, D. and M. Rein 1994. Frame Reflection. New York: Basic Books.
Schwarz-Shea, P. 2006. Judging quality. Evaluative criteria and epistemic communities. In
D. Yanow and P. Schwartz-Shea (Eds), Interpretation and Method. Empirical Research Methods and the Interpretive Turn. London, New York: Sharpe, pp. 89113.
SER Advies 01/01 2001. Levensloopbanen: Gevolgen van Veranderde Arbeidspatronen, Den Haag:
SER.
Stone, D. 1988. Policy Paradox. The Art of Political Decision Making. New York: W.W. Norton.
Terpstra, D., and M. Oostveen. 2001. CNV: CDA-Voorstellen Veelbelovende Voorzet. CDV 22
(7/8/9), pp. 9598.
Van der Braak, J.W. 2001. VNO-NCW: Arbeidsmarkt verruimen en capaciteit van mensen beter
benutten. CDV 22 (7/8/9), pp. 176179.
Westerlaken, A.A. 1996. Maatwerk vereist voor werknemer met zorgtaken voor kinderen. CDV
16 (7/8), pp. 414419.
Wodak, R., R. de Cillia, M. Reisigl and K. Liebhart. 1999. The Discursive Construction of National
Identiy. Edinburgh: Edinburgh University Press.
Yanow, D. 2000. Conducting Interpretive Policy Analysis. Newbury Park, CA: Sage.
Yanow, D. 2006. Thinking interpretively: Philosophical presuppositions and the human s ciences.
In Interpretation and Method. Empirical Research Methods and the Interpretive Turn.
London: Sharpe, pp. 526.
chapter 10
Between the Union and a United Ireland

Shifting positions in Northern Irelands
post-Agreement political discourse
Laura Filardo-Llamas
University of Valladolid
This chapter concerns the double objective of political discourse: spreading

ideological beliefs and allowing political progress. Political discourse can be
effective by means of carefully chosen linguistic elements (pronouns, lexical
referents, metaphors). If successfully constructed, the discourse can live on
for many years, as we see in the Northern Irish Agreement of 1998 that is still
resounding in current debates. Its ambiguity and vagueness allowed for different
interpretations, in such a way that the support, or rejection, of the agreement
could be legitimised, depending on the ideological beliefs of the community. An
analysis of selected instances of its discourse over time shows how the ideological and political positions that are discursively constructed in such a way that
they are close to both the producers and the interpreters of the document, but
also to the political process of production and interpretation in which the texts
are embedded. Text World Theory and Cognitive Linguistics are the tools for
analysing the different types of mental representations that are recalled by each
instance of discourse.
Introduction
This chapter proposes a range of linguistic categories as a tool for looking at the
way in which political positions have been discursively created in post-Agreement
Northern Ireland (NI). We set out from the notion that contemporary political
discourse (PD) is meant to address multiple audiences at the same time (Achugar
2008:171). This involves the employment of several discursive strategies that do
not only allow the designation of different socio-political groups, but which also
contribute to fulfilling the politically-advancing function of any instance of PD.
Therefore, it could be argued that the discursive construction of multiple audiences is a consequence of the double objective of PD. On the one hand, it is aimed
208 Laura Filardo-Llamas
at discursively producing and reproducing a social and political construct of reality (Achugar 2008; Berger and Luckman 1966). On the other, it has a purely
political function, as it is a constitutive part of political action. Both functions
can be linked to the different elements of a discourse practice, as identified by
Fairclough (1989): a text, a context of production and interpretation (the immediate political activity), and the wider sociocultural practice of production and interpretation (the members of the communities at whom the speeches are addressed).
Taking into account this double function, it is necessary to consider that the
strategic use of language in PD is closely connected to the political principles
underlying discourse in the place where this is produced. In the NI context, it is
particularly important to consider how ideology and the political practice blend
(Hayward 2010), as it could be argued that ideological beliefs have determined
the conflict situation in NI, which has recently been transferred to the political
arena. This particular feature of the NI sociocultural practice stresses the political
significance of ideology and identity.
The concepts ideology (defined as the basis of the social representations
shared by members of a group, Van Dijk 1998:8) and group identity (the mental representation through which an individual considers the self a member of a
particular group, ibid.: 120), are closely interwoven, and they stress the need to
acknowledge social cognition as an important component of the discourse practice. Social cognition serves to explain the way in which a relationship can be
established between discourse and society, and particularly between PD, political
practice and the transmission of ideology. Therefore, it could be argued that the
ideologies shared by each of the NI communities monitor their perception about
society, and they have an impact on the way that the PD associated with each of
the communities is constructed. This double and mediating relationship between
PD as action and PD as discourse is illustrated in Figure 1.
Social cognition is important because it has an impact both on the context of
production (the participants immediately involved in the communicative situation) and the context of interpretation (the (un)intended audiences). This involves
the strategic use of language in such a way that any speech would receive the
desired response from the perspective of the participants immediately involved
in the political communication as well as from the rest of society, as is also shown
in Elevelds chapter on unusual coalitions (this volume).
This strategic exploitation of discourse is easily achieved in NI, where current
political debates stem from the 1998 Agreement. The vagueness of the language
used in this document (Alonso 2001) allows NI politicians to discursively legitimise (Chilton 2004; Van Leeuwen and Wodak 1999) different political actions in
relation not only to their own ideological beliefs but also to the socio-political
Chapter 10. Shifting positions in Northern Irelands post-Agreement political discourse 209
Context of production
(context of situation)
Social practice:
Political discourse
action
Context of interpretation
(society)
Social practice:
Political discourse
justification of action or not
Social cognition:
Individual meanings +
group representations
Social cognition:
Individual meanings +
group representations
Discourse practice:
Political discourse
ideology transmission
Discourse practice:
Political discourse
ideology spread or not
Figure 1. Mediated relationship between discourse and a social practice

in political communication.
process in which they are embedded. The term Agreement subsequently becomes
an empty signifier (cf. Montessano Montessori, this volume; Laclau 1996/2007)
whose meaning can be constructed in different ways by each of the discourse
participants.
Given the double political and discursive function that is ascribed to PD, this
chapter also has a twofold objective: looking at the spatial, temporal and sociopolitically-motivated evolution of the different discursive representations constructed by some NI politicians, and relating them to given social and political
positions. This requires identifying the mental representation that can be triggered
by both the directly and indirectly intended audiences in a way which may help
us understand how current political communication seems to employ a double
discursive strategy which could address different groups in the audience at the
same time. A bottom-up linguistic method is used in order to elicit what actually
happens in the speech acts.
Context and data

The 30 years of overt conflict in NI apparently came to an end in 1998, with the
Belfast/Good Friday Agreement, a document which was signed by the British
and Irish governments and subscribed by most of the NI political parties, with
the exception of the Democratic Unionist Party (DUP). The ambiguity and vagueness of the language used in this accord has not only allowed a gradual transition
from overt conflict to, at least, negative peace, understood as the containment of
war and the lack of physical conflict (Wenden and Schffner 1995: xix), but also
a construction of reality which seems to reflect a paradoxical reality (Aughey
2002:2; Filardo-Llamas 2009). Nevertheless, it was that vagueness which made it
difficult to justify it because it required uttering a political discourse which could
be interpreted in very different ways at very different levels (Filardo-Llamas 2009,
2010; Hayward 2010).
The semantic emptiness of some of the signifiers (Montesano Montessori,
this volume) used within the Agreement made it difficult for its provisions to
be implemented, particularly in relation to issues such as the decommissioning of weapons by paramilitary groups, and the required changes in policing,
both of which were closely related to the political actions and beliefs of Sinn
Fin (SF) and the DUP.1 These problems seemed to be overcome politically in
2006, when the St Andrews Agreement was reached. Within this new document those two controversial aspects were dealt with, and the DUP (the party
who had most thoroughly opposed republicans being included in government)
agreed to power-sharing with SF in the NI Executive. At the same time, SF (the
representative of the community who had most strongly opposed a new police
force by relying on an accusation of a mere change of name) fully accepted the
new Police Service of Northern Ireland (PSNI), and agreed to nominate party
members to its Policing Board.
This development of events apparently suggests a change in policy by the
DUP and SF, at least in relation to the status issue: decommissioning and policing. Nevertheless, a thorough linguistic analysis of their discourses proves that
this might not exactly be the case. While in 2006 their ideological beliefs are still
retained, their political actions are no longer justified in relation to those shared
beliefs, but to the actual political practice in which they are embedded. A change
in the circumstances, therefore, involves a change in the way their discourse has
been constructed.
1. Both the DUP and SF are chosen because they represent the most extreme parties in mainstream NI politics.
With these premises in mind, five speeches were analysed not only to reveal
the difference in ideological beliefs and their portrayal of a paradoxical reality,
but also a diachronic difference in the discursive construction of NI politics.
Three of the speeches are a response to the 1998 Agreement, made by the leaders of the most prominent unionist parties: David Trimble of the Ulster Unionist
Party (UUP), Ian Paisley of the DUP and the leader of the Republican Party (SF):
Gerry Adams. The other two speeches are Adams and Paisleys reactions to the St
Andrews Agreement.
Methodology
Given the interdisciplinary objective of this paper and assuming that there is a
double function of PD that relates to the intended social and political audiences,
I have followed an eclectic approach which takes elements both from cognitive
linguistics and the socio-political sciences. Following Critical Discourse Analysis
(CDA), it is argued that the relationship between language and society can be partly
explained by looking at the linguistic mechanisms of persuasion through which
social representations of reality are reproduced and maintained, and, hence, legitimised (Chilton 2004). The construction of a social representation of reality can be
used to explain conflict, inasmuch as the legitimation process can be explained as
the construction of symbolic universes (Berger & Luckman 1966:110122).
The double ideological and political function of PD can be explained by relying on at least a double legitimisation: that of political action and that of a particular view of society (cf. Filardo-Llamas 2009). Thus, legitimisation can be defined
as the promotion of a specific representation of society which is imbued with
evidence, authority and truth, in such a way that the discursive construction of
society and a proposed political practice are objectivised (Chilton 2004:23). In
this perspective, Text World Theory (Gavins 2007; Werth 1999) can be a useful
tool to explain the linguistic construction of these multiple representations. In this
theory, social cognition is taken into account as a determinant of a broad conception of context, being not only the immediate context of a situation, but also the
social structure within which a speech is uttered. That social structure influences,
in turn, the knowledge that discourse participants have, including their shared
background knowledge (Gavins 2007:8), which is grounded in their personal,
interpersonal, group, institutional, national and cultural experiences (Van Dijk
2005:7778). All of them have a bearing on the mental representations evoked
by a given instance of discourse. These mental representations are known as text
worlds (Werth 1999:17), or discourse worlds in Chiltons (2004) adaptation of the
theory to the study of PD. This focus on text worlds and the qualitative analysis
of discourse complements the quantitative dimension of the word-space model

proposed in Dahlberg and Sahlgren (this volume).
The immediate context of a situation is equally important, and it is argued
in this chapter that depending on the participant involved in the actual practice
of PD, discourse can yield a different type of text world, i.e., a different conceptual representation. These text worlds frequently overlap and blend, and are
therefore a useful tool for explaining how different audiences can be addressed.
Following traditional communication theory and functional approaches to the
study of language (Halliday 2004), it can be argued that communicators rely on
the contextual variable tenor: i.e. that which explains who is involved in communication, which are their social roles, and which type of distance is established
between them. In the immediate context situation, the tenor would be the politician uttering the speech and another politician who listens to it and is expected to
react to it. However, there are also less immediate social participants, such as the
members of the social group the politician represents. Likewise, the social relationship that is established between the double intended audience is not similar,
inasmuch as politicians (speakers as well as hearers) can be considered equals
on the social scale, although we usually attribute authority to them in relation to
the rest of society.
The double audience influences the types of legitimisation which can be identified, depending on whether they are related to the objectivisation of ideological beliefs or of given political actions and their effects in the political process
(Filardo-Llamas 2009). Objectivisation can be linked to the discursive construction of political text worlds which are more likely to be uncovered by fellow politicians, whereas ideology can be contained in ideological text worlds. Although
both types of text worlds are discursively blended and they operate at the same
time, this distinction can help us understand the complexity that underlies political communication, and the, at least, double function that can be attributed to it
(see Figure 2).
Paul Werth (1999:51) defined a text world as a deictic space, defined initially
by the discourse itself, and specifically by the deictic and referential elements in
it. In this definition the importance of linguistic choices is already hinted at, as
they become a cue for text-world construction and triggering. Text World Theory
is complemented by the Cognitive Grammar concept of grounding (Langacker
2008:259) in explaining the relationship between the actual context of a discourse
and the elements that occur in the discourse itself. Both Chilton (2004) and Werth
(1999) stress that language should be understood in spatial terms: that linguistic entities reflect a distal relationship to the speaker. These indexed entities may
refer to participants, space, and time. Together they construct a text world, and
therefore become world-defining elements, which specify the salient entities
Context of production (immediate situation)

Social practice:
Political discourse justification
of action or not
Social cognition:
Individual meanings + group
representations
Discourse practice:
Political discourse ideology
spread or not
Participants in PD practice
Audience politicians
Political text world
Uttering politicians
Society
Ideological text
world
Context of interpretation (society)
Figure 2. Political functions and the development of text worlds.
of the text world by lexical choice2 and locate them in a deictic relationship to
the speaker. Such linguistic elements of proximity or distance can be pronouns,
adverbs, or verb tense.
Relationships can be established between the entities that define a text world,
as through language we represent a world in which participants are involved in a
process, in given circumstances (Halliday 2004). Participants and circumstances,
as world-defining elements, can be revealed by analysing deictics and reference,
but if we want to understand how political actions are legitimised, we also need
to take into account the processes in which participants are involved. Our mental
representations may vary depending on whether a politician is portrayed as being
involved in a verbal process (e.g. promising) or in a material one (e.g. implementing). Thus, it is important to take into account the type of verbs which are associated to each discourse participant. We have done so by following Hallidays (2004)
analysis of transitivity, involving four kinds of processes: material (doing), verbal
(saying), mental (sensing), and relational (being) processes. The analysis of verbal
2. The importance of presence is highlighted because it seems that not mentioning something
in an instance of discourse could deny its existence. Therefore, in conflict situations, problems
can be obviated by not mentioning them.
elements can help us understand how participants advance in the discursively

created mental representation of a given political reality (Werth 1999:57).
Given that PD is inherently persuasive, it is also important to analyse linguistic strategies that can help us understand how ideology is spread and action is
legitimised in an unconsciously effective way. Metaphors are of key importance
here, because, they have a mediating role between conscious and unconscious
means of persuasion, that is, between cognition and emotion (Charteris-Black
2005:1320). Metaphors are instrumental in the creation of a moral perspective
of life that can be easily associated with the double function of political discourse,
as both actions and beliefs can be evaluated. The persuasive quality of metaphors
stems from their ability to establish a set of connections among a source space, a
target space and a blended space (Langacker 2008:51). By using metaphors, we
can conceptualise an abstract aspect of society, such as politics, in terms of another
aspect of society which can be more easily understood, for example war. This creates a blended image in which the properties of the source domain war are attributed to those of the target domain politics. This device is not only persuasive, but
it also represents an advantage for politicians, who cannot be held responsible for
any interpretation of such oblique (or empty) statements, as any interpretation is
embedded and biased by the recipients text world.
Aspects of language use as evidence of shifting positions

Shifting positions in Gerry Adams 1998 speech
Gerry Adams 1998 speech provides a very interesting example of the use of pronouns to address different audiences. By including or excluding discourse
participants, Adams addresses different groups of people in the same speech. This
is particularly the case if we look at how the pronoun we refers to six categories
of inclusive groups:
1. SFs negotiating team
2. the members of the leadership of SF
3. the members of SFs Ard Fheis3
4. republicans
5. SF and republicans, and
6. republicans and unionists
3. The Ard Fheis is the Gaelic term used to refer to the annual party conference of many Irish
nationalist political parties.
The different referents, whose appearance in the speech can be related to the creation of different types of text worlds, are identified by looking at the co-text in
which the pronoun appears.
Pronouns that focus on the role of SF, referring to the negotiating team or
to the members of the party, serve to justify given political actions or thoughts
which are discursively located near the speakers deictic centre, and in opposition to those of the others (Example 1). At the same time, the social audience is
addressed with the pronoun you (Example 2), hence attributing it a key function
in evaluating the positive role of SF at the negotiation talks and the subsequent
Agreement.
(1) Our concerns as we approached this intensive period were that the two
governments had yet to agree on many of the substantive issues [] We were
also concerned that the officials at the British end [] would once again take
up a unionist line.
(Adams 1998)4
(2) Our party had argued for that type of intensive, concentrated and focused
dialogue. In fact, you will recall it was we who first asked for a timeframe for
the negotiations.
(Adams 1998)
On the other hand, the more inclusive use of the pronoun we helps in spreading
ideological beliefs, particularly when indexing the republican community in general, and they can be found in the near co-text of historical references. In that way,
not only beliefs about the desire of a United Ireland are spread but they are also
justified by connecting them to historical events. This seems to result in an image
where different events (1998 and 1978; 1998 and 1916) are conceptually blended
(Fauconnier & Turner 2002) by highlighting the similarities (a clamour of freedom) and pointing out the continuation of both beliefs and struggle throughout
time (Example 3).
(3)
82 years ago on this very day this city echoed to the clamour of freedom as the
men and women of 1916 declared the Republic. Now as the century draws to
a close and on the culp of the millennium that self same clamour echoes all
around us.
(Adams 1998)
It is interesting to see how unionists are incorporated here as a relatively close

participant in the political text world by using the pronoun we following a first
person pronoun I and a mental verb (Example 4). In this way, we find unionists
as characters of a positive desired ideological text world (Werth 1999:227233)
which can be achieved if all the political participants engage in particular actions.
4. Formatting in the examples has been added for purposes of emphasis.
By means of this mental representation the political role of unionists is legitimised

on the grounds of certain beliefs about the future.
(4) I would therefore like to take this opportunity to spell out to the northern
protestant and unionist community the core political values that lie at the centre
of our wish to engage with you. [] There is a common need: to recognise the
integrity of the other; to be at peace with each other; to understand the way
we have hurt one another; [] to find our common ground; to celebrate our
difference as diversity. And as equals.
(Adams 1998)
Other strategies, particularly the use of metaphor, shall be noted in Adams

speech. The struggle is a journey mega-metaphor (Gavins 2007:151) that
runs throughout the speech does not only contribute to the creation of a political text world where the Agreement is another stage in the ongoing journey
towards a United Ireland, hence justifying that action, but it also connects
that image to the beliefs that are shared by the broad republican community
(Example 5). In this way, Adams builds an image which can be shared not only
by those republicans who agree with the document but also by those who oppose
it because they consider it a final settlement. Besides, the constant references to a
United Ireland, and this and our country throughout the speech help to locate
this place at the deictic centre, which is shared by the speaker and the intended
broad republican audience.

(5) For now, I want to encourage you all to give your views in an open, frank and
comradely way about where you think we are, where our struggle is, how last
weeks developments fit into this and how we move from this point forward
towards our goal of unity and independence.
(Adams 1998)
The same conceptual metaphor underlies a political text world in which republicans are advancing in their own struggle as well as in the political negotiations
leading to the Agreement (Example 6). SFs effort is emphasised through a comparison with the negative image of those unionists who blocked the process and
prevented a political and republican advancement.

(6) Meanwhile back at the Talks venue, the Unionists were still blocking and
impeding progress.
(Adams 1998)
Adams constant shift between political and ideological text worlds, and the consequent blend between them, serves to legitimise the 1998 Agreement by relying
on shared ideological beliefs, which are used as the basis for justifying SFs political actions, not only in front of the republican community, but also in front of the
other members of Northern Irish political life.
Shifting positions in unionists 1998 speeches

Unionist position shifting can be found in each instance of discourse, and in the
way society is portrayed by each of the analysed politicians: David Trimble and Ian
Paisley. In general, different types of text worlds dominate the speech of each of
them, mainly because of their different political objectives. Most of the text worlds
in Trimbles speech are political in nature, due to his attempt to justify his partys
political involvement in the negotiation talks and the subsequent acceptance of the
Agreement. By contrast, Ian Paisleys speech invokes tightly interwoven ideological and political text worlds.
The strategic combination of an exclusive we with a unionist-based journey
metaphor serves to create a political text world where the Agreement is legitimised
in terms of its connection to past historical events. In this way, Trimble establishes parallelisms between previous positively evaluated historical actions and
his own actions; something which is ideologically connected to the unionist desire
to maintain the constitutional status of NI as part of the UK (Examples 7 and 8).
Even if the participants that form part of this text world are political figures, they
share ideological beliefs with the unionist socio-cultural group. Thus, by establishing a metaphorical link between the positive past and the present, Trimble is
trying to legitimise his political actions to a double audience: Unionist politicians
who oppose the Agreement, and the unionist community.
(7)
This Agreement is the culmination of a process begun by the current leader
of the DUP and my predecessor, Lord Molyneaux, in 1988 when they wrote
to the then Secretary of State, Tom King. We have succeeded in replacing the
Anglo-Irish Agreement with a British-Irish Agreement, as proposed separately
by the UUP and the DUP in the 1998 Talks.
(Trimble 1998)
Trimbles speech is full of temporal references because historicism is the key connection between the legitimisation of the 1998 Agreement and unionist beliefs.
These temporal references establish world-shifts (Gavins 2007:4650), which contribute to creating an image where the present is blended with the past, as both
of them depart from the speakers central perspective. Thus, the appeal to historical events is something that both politicians and the broader social audience can
trigger and identify with, particularly when presenting an opposition between an
undesired past and future circumstances based on the Agreement. Changes in the
degree of inclusiveness of we, and the combination of the most inclusive uses with
the noun phrase the people of Northern Ireland result in the characterization
of both intended audiences as a single participant in the socio-political text world
that is used to represent the outcome of the Agreement (Example 8). The blended
present-past temporal frame together with the use of a politics is economy
metaphor do not only serve to evaluate positively the political role of the UUP,
but also to present a positive socio-political present and future. As a result both
the actions of the UUP and the Agreement are legitimised.

(8) What mattered most to UUP negotiators above all else, was that the Talks
process be used as a vehicle to strengthen the Union between Great Britain
and Northern Ireland. For the first time since 1972, the people of Northern
Ireland are to have the democratic deficit removed. For the first time in 26
years it will be for the people of Ulster to determine their future. No longer
will we be at the whim of Secretaries of State who neither care nor understand
our Province.
(Trimble 1998)
The vagueness of the Agreement allows unionist politicians to represent a completely different reality, as it both strengthens (David Trimble, Example 8) and
weakens (Ian Paisley, Example 9) the Union. Besides, they use different strategies,
with David Trimble discursively creating a political text world, and Ian Paisley
constructing a negative future socio-political text world whose negative evaluation
is partly the consequence of an intertextual appropriation of Adams metaphorical
expressions and linguistic strategies (Example 9).
(9) The reaction of ordinary Unionists is one of outrage and amazement that any
unionist leader could set his hand to such a deal which so fundamentally
weakens the Union and which would place the Province inexorably on the
road to a United Ireland.
(Paisley 1998)
The journey metaphor helps to create an image of a future political text world
which is diametrically opposed to unionists beliefs and desires, hence delegitimising the deal. This type of blended negative political and ideological text world
abounds in Paisleys speech, which is full of references to a United Ireland, aimed
at addressing unionists fears of living in a United Ireland. By placing the unionist
community in a passive role, that is, as suffering the consequences of the actions
of others and removing their active role, the certainty of unionist fears is stressed.
Thus, the DUP is legitimised, as it is the party promoting discursive accessibility
to those negative future worlds. This becomes clear by doing a close linguistic
analysis of the context of those future worlds which are frequently subordinated to
the reaction of politicians (Example 9) or to them being made visible by Paisley
and other members of the DUP (Example 10).
(10) We are determined that the people will hear the facts. We will be joining
with other like-minded unionists and loyalists to put the case for the Union.

(Paisley 1998)
The DUP is constructed here as the source of political and moral authority, and
their representation of negative future ideological text worlds in Example10 is
related to the need of justifying their negative political actions, i.e. their saying no
to the Agreement. The subordination of negative future ideological text worlds to
present political actions (the Agreement) contributes to the negative evaluation of
those actions, which are consequently delegitimized (Example 11).
(11) In one of the most despicable aspects of the deal, all terrorist prisoners are to
walk free within 2 years whilst the RUC is to be put on the altar and sacrificed
to keep the IRA happy.
(Paisley 1998)
The prominence of ideological text worlds throughout Paisleys speech is related

to the need to justify their political actions, while delegitimising the actions of
other not-ordinary unionist politicians, as we can see in Example(9). The double
audience thus receives the message, and pro-Agreement politicians obtain a no
response coming both from Paisley and part of the unionist community based on
what were supposed to be shared unionist ideological beliefs.
Shifting positions in responses to the St Andrews Agreement

The analysis of responses to the renegotiation of the Agreement at St Andrews
in 2006 shows how the apparent discursive reinterpretation of facts is important
for social change (see also Eleveld on unusual coalitions, this volume). SFs and
the DUPs reactions to this document shows an apparent shift from the two most
extreme positions to more mainstream ones. This is done by deploying similar
linguistic strategies with a slightly changed value.
The political text world which is portrayed in both Adams and Paisleys
speeches is no longer characterised by a division between political participants,
but by a shared commitment between the representatives of both communities. In
Adams political text world, the commitment of unionists is represented by means
of a desire text world (Werth 1999:227233) where unionists and republicans
work together. This type of subordinate text world highlights Adams positive commitment (I want) to this event. Besides, this new political reality, depending on a
desire, is a useful strategy for addressing all the intended audiences regardless of
their ideological beliefs. The positive traits of the subordinated political text world
are stressed through the good governing is creating metaphor (CharterisBlack 2005:124), and its association of the material action building to politicians
belonging to both communities (Example 12).
(12) I want to appeal to unionists to come at this in a way which looks at the
differences which we have been able to put aside and the potential which we
now have to build a new Ireland where all the children are treated equally.

(Adams 2006)
Adams use of pronouns is once again significant, particularly in the case of we,
because its referent changes depending on the intended audience each section
of the speech is addressed to. However, there is less variation in this speech than
there was in 1998. Not only can more inclusive references be found, but those
deictics indexing the whole republican community are more frequently mixed
with global inclusive references, where all those possible actors that take part in NI
political life are embraced, regardless of whether they are politicians or not. Both
of these strategies result in blended ideological and text worlds which are not only
characterised by a diluted political opposition, but also by a somewhat diffused
ideological one, as both republicans and unionists can share the same imagined
homeland: Ireland (Example 13).
(13) We are Irish republicans. We believe in Irish unity and the coming together
of orange and green. We believe in peace. All of us are going to be challenged
and that includes the two governments. They cannot absolve themselves of
responsibility in all of this.
(Adams 2006)
Nevertheless, even if both audiences are included in Example (13), the change in
transitivity patterns accompanying the pronoun we is a significant one. When it
is employed with a republican referent, relational and mental processes are highlighted, whereas the purely inclusive we is the object of an implicit, unknown
agent. That gives some prominence to republican beliefs as we cannot forget that
they are still Adams main intended audience, although the verbal shift and the use
of the passive voice still stress an important role to be shared by all socio-political
participants.
As in 1998, the main acting position in Paisleys political text world is occupied by the DUP who has performed positive material and verbal actions such
as demanding or achieving progress (Example 14). The positive evaluation stems
from the journey metaphor whose destination (the Union) has changed in 2006,
hence establishing a clear link between the DUPs political action and unionist
beliefs. By doing this, Paisley legitimises the renegotiated Agreement, and his
partys role, to the broad unionist community as well as other politicians.
(14) The Democratic Unionist Party has made considerable progress throughout the
course of these talks on the important issues facing us at this time. Unionism
can have confidence that its interests are being advanced and that democracy
is winning the day.
(Paisley 2006)
Unlike in 1998, the most frequently used pronoun in this speech is we. Although
there are cases when the referent of this pronoun is ambiguous (Example 14) and
it could include the whole unionist community, or exclude it in favour of only
unionist politicians, exclusive uses of the pronouns are more prominent. Thus,
even if the linguistic strategy has changed slightly, Paisleys objective with this
speech remains the same as in 1998 and presents a political party that has both
epistemic and deontic authority over the community it represents. Besides, the
rare uses of the pronoun they, and the lack of pronominal opposition, downplay
the 1998 discursive antagonism between the DUP and republicans, who, at this
time, occupy a shared political space (Examples 15 and 16).
(15) The DUP has been consistent in our demand that there must be delivery
from the republican movement before devolution can be restored in Northern
Ireland. The days of gunmen in government are over.
(Paisley 2006)
(16) As we have consistently said, it is deeds and not deadlines that count.

(Paisley 2006)
Furthermore, there are times when an implicit justification of at least some of

the members of the republican movement can be revealed (Examples 15, 16).
By allowing SF representatives in Government who had previously been rejected
because of alleged membership of the IRA, Paisley implicitly acknowledges the
IRAs decommissioning of weapons in September 2005. He is creating a blended
political and ideological text world in which none of the members of the NI executive belongs to paramilitary organisations. An implicit criticism of other unionist
parties can be observed in the same examples: the word deeds being placed
before deadlines seems to refer to Trimbles problems throughout the implementation process due to the IRA not decommissioning whenever it was required, and
this resulting in several suspensions of the NI Assembly. With very subtle linguistic choices, Paisley manages to justify an apparent shift in his political position,
as he is now prepared to share government with SF. The justification is realised by
creating a political text world in which his political actions are the consequence of
the political actions of others, without this resulting in any change in the underlying unionist-based ideological text world.
Concluding thoughts
The analysis presented in this chapter gives evidence for the double objective of
PD. The linguistic findings show how language is used in such a way that all the
intended audiences are simultaneously addressed in one single instance of discourse. All the analysed speeches have a double audience, including both society
and the other politicians in NI. Nevertheless, the understanding and the textual
construction of those audiences varies depending on the time when the speech
is uttered. The prominence of ideological text worlds in the speeches produced
in 1998 stresses the importance of those social representatives that share certain
ideological beliefs as common ground with the political speaker. This involves not
only using inclusive language, such as pronouns and referential expressions, but
also contextually placing those references near their most intrinsic shared beliefs,
such as the Union or a United Ireland. The shared beliefs allow the creation of certain ideological text worlds, with which a person having the same values can identify (Gavins 2007) and possibly form unusual coalitions (Eleveld, this volume).
By contrast, later speeches are not as closely focused on addressing a section
of society, but on addressing society as a whole. The shared beliefs of each community still appear in every instance of discourse, but their importance is somewhat downplayed as politicians need to reach not only those who already support
them, but they also need to create a symbolic universe (Berger & Luckman 1966)
that will include other people. These slightly modified ideological text worlds are
the ones that allow political conflict to be resolved by faintly redefining the sociopolitical groups which are designated by each politician.
The changes in the political text worlds that can be uncovered in the two latter
speeches also aim at conflict resolution (Filardo-Llamas 2008). Because the politicians that utter them need to make advances in the political process and reinforce
their position, the most significant shifts are related to the discursive construction
of those political text worlds. Minor linguistic adjustments, such as the re-development of metaphors, modalisations, or changes in deictic variables, allow a different
construction of political life in NI. This significantly different discursive construction, whose main feature is the re-evaluation of formerly opposed discourse participants in political text worlds, is the one that has allowed the advancement of
the political process in NI, showing an effective political role of PD.
The analysis gives validity to the linguistic discourse approach of Text World
Theory combined with CDA, and shows how current PD is used to address multiple audiences, and how audience-variation is intrinsically connected to the functions of PD. Likewise, the analysis of these linguistic and contextual elements can
help us gain some insight into how conflict can be discursively reproduced and
managed, as subtle and minor discursive changes may construct a new symbolic
universe which can be eventually legitimised.
Data
Analysed speeches
Adams, G. 1998. Presidential Address to Sinn Fin Ard Fheis. 18 April.
Adams, G. 2006. Comments at St Andrews after the End of Three Days of Talks. 3 October.
Paisley, I. 1998. Statement at the Launch of the DUP No Campaign. 15 April.
Paisley, I. 2006. Comments at St Andrews after the End of Three Days of Talks. 3 October.
Trimble, D. 1998. Speech to the Northern Ireland Forum. 17 April.
References
Achugar, M. 2008. What We Remember. The Construction of Memory in Military Discourse.
Amsterdam: John Benjamins. DOI: 10.1075/dapsac.29
Alonso, R. 2001. Irlanda del Norte. Una Historia de Guerra y la Bsqueda de la Paz. Madrid:
Editorial Complutense.
Aughey, A. 2002. The art and effect of political lying in Northern Ireland. Irish Political Studies
17(2), pp. 116. DOI: 10.1080/714003199
Berger, P. and T. Luckman. 1966. The Social Construction of Reality. A Treatise in the Sociology
of Knowledge. London: Penguin.
Charteris-Black, J. 2005. Politicians and Rhetoric. The Persuasive Power of Metaphor. New York:
Palgrave Macmillan.
Chilton, P.A. 2004. Analysing Political Discourse. Theory and Practice. London: Routledge.
Fairclough, N. 1989. Language and Power. London: Longman.
Fauconnier, G. and M. Turner. 2002. The Way We Think. Conceptual Blending and the Minds
Hidden Complexities. New York: Basic Books.
Filardo-Llamas, L. 2008. A comparative study of the discursive legitimation of the agreement by
the four main Northern Irish political parties throughout time. Ethnopolitics 7(1), pp.21
42. DOI: 10.1080/17449050701846634
Filardo-Llamas, L. 2009. Proposal for the analysis of the legitimatory function of political discourse in the Northern Irish context. In I. Iigo-Mora et al. (eds.), Analysing Political Discourse Strategies. Newcastle: Cambridge Scholars Publishing, pp. 135152
Filardo-Llamas, L. 2010. Discourse worlds in Northern Ireland: The Legitimisation of the 1998
Agreement. In K. Hayward and C. ODonnell (eds.), Debating Conflict and Peace: Dissonant
discourses in Northern Ireland. London and New York: Routledge, pp. 6276
Gavins, J. 2007. Text World Theory. An Introduction. Edinburg: Edinburgh University Press.
DOI: 10.3366/edinburgh/9780748622993.001.0001
Halliday, M.A.K. 2004. An Introduction to Functional Grammar, 3rd edition. (Revised by
Christian M.I.M. Matthiessen). London: Edward Arnold.
Hayward, K. 2010. The role of political discourse in conflict transformation: Evidence from
Northern Ireland. In K. Hayward and C. ODonnell (Eds.), Debating Conflict and Peace:
Dissonant Discourses in Northern Ireland. London and New York: Routledge, pp. 115.
Laclau, E. 2007 [1996]. Emancipation(s), London: Verso.
Langacker, R. 2008. Cognitive Grammar. A Basic Introduction. Oxford: Oxford University Press.
Van Dijk, T.A. 1998. Ideology. A Multidisciplinary Introduction. London: Sage
Van Dijk, T.A. 2005. Contextual knowledge management in discourse production: A CDA perspective. In R. Wodak and P. Chilton (eds.) A New Agenda in (Critical) Discourse Analysis.
Amsterdam: John Benjamins, pp. 71100.
Van Leeuwen, T. and R. Wodak. 1999. Legitimizing immigration control: A Discourse-Historical
Approach. Discourse Studies 1(1), pp. 83118. DOI: 10.1177/1461445699001001005
Wenden, A.L. and C. Schffner. 1995. Introduction. In C. Schffner and A.L. Wenden (Eds.)
Language and Peace. London: Routledge, pp. xvxxv.
Werth, P. 1999. Text Worlds: Representing Conceptual Space in Discourse. Harlow: Pearson
Education.
chapter 11
Systematic stylistic analysis

The use of a linguistic checklist
Maarten van Leeuwen
Leiden University Centre for Linguistics, Department of Dutch
Attention to style in political discourse with a focus not on what political actors
say, but on how they say it, can mainly be found within the research traditions
of rhetorical criticism (RC) or rhetorical stylistics (RS) on the one hand, and
critical discourse analysis (CDA) on the other hand. The tradition of CDA
has been criticized for being ad hoc: analyses often fail to clarify why on the
linguistic level some linguistic means are analysed, and others not. Within both
frameworks of RC/RS and CDA, it has been suggested to use a checklist as
a methodological tool in stylistic analysis, but this has barely been put into practice. In this chapter I discuss what the use of a checklist can yield for stylistic
analysis of political discourse and illustrate this with a detailed stylistic analysis
of a speech held by the Dutch controversial politician Geert Wilders.
1. Introduction1
Since classical antiquity, much attention has been paid to the question how politicians and other public speakers can effectively construe their message. Attention to
style in political discourse (i.e., a focus not on what political actors say, but on how
they say it) can mainly be found within two different research traditions that are
characterized by an enormous heterogeneity in both objects of study and methodology. Rhetorical criticism (Foss 2004) or rhetorical stylistics (Fahnestock
2009:195) stems from a tradition that goes back primarily to the work of the classics, like Aristotle, Cicero and Quintilian. The other tradition involving stylistics
takes on various approaches from discourse analysis (see Fahnestock 2009 for an
overview), especially in critical discourse analysis (Fairclough 2001; Wodak 2006;
1. This chapter was written in the context of the NWO sponsored project Stylistics of Dutch
(www.stylistics.leidenuniv.nl). I would like to thank Ton van Haaften, Jaap de Jong, Arie
Verhagen, Ninke Stukker and Suzanne Fagel for their helpful comments on an earlier version
of this paper.
226 Maarten van Leeuwen
Wodak and de Cillia 2006) that leans heavily on systemic-functional linguistics

(Halliday and Matthiessen 2004).
Critical discourse analysis has been criticized for being ad hoc (cf. Poole
2010): analyses often fail to clarify why, on the linguistic level, some linguistic
means are analysed, while others are not. In such research, it looks as if a selective
choice of linguistic elements is made which supports the authors interpretation
(Van Rees 2005:96). Within the frameworks of both critical discourse analysis
and rhetorical criticism, it has been suggested to use a checklist as a methodological tool in stylistic analysis (cf. Corbett 1990; Fahnestock 2009; Fairclough 1992,
2001; Fowler 1991; Fowler and Kress 1979). Given the methodological critique, it
is striking that these checklists are barely used in practice.2
In this chapter the use of a linguistic-stylistic checklist will be put to the test
to establish to what extent it is a useful methodological tool for stylistic analysis.
A detailed stylistic analysis of a speech held by the Dutch controversial politician
Geert Wilders serves an example. Geert Wilders has attracted much attention
from the media in recent years. He was the leader of the right-wing populist Party
for Freedom (PVV), and the main point on his political agenda is to stop what he
calls the Islamification of the Netherlands. Internationally he is mostly known for
his anti-Islam movie Fitna, which focuses on the assumed threat and barbarism of
the Islam. After the national elections in June 2010, the Party for Freedom became
the third political party in the Netherlands.
In this chapter I will present a detailed stylistic analysis of one of Wilders most
discussed speeches: the speech he held in Dutch Parliament during a debate on
Islamic activism in 2007. In this speech, Wilders incited a ban on the Koran, and
argued that what he calls the Islamification of the Netherlands has to be stopped.
The speech caused quite some commotion, especially because Wilders called the
then Minister of Integration, Ella Vogelaar, insane. The speech is still representative for the way in which Wilders presents himself in addresses: with radical views,
breaking with political etiquette, and in populist wordings which can intuitively
be described as clear.
In the next section I will describe the method and the checklist used; the stylistic analysis will be presented in Section 3. In the Conclusion (Section 4) I will
come back to the question to what extent a checklist is a useful methodological
tool for stylistic analysis of political discourse.
2. CDA has not only been criticized for being ad hoc methodologically (i.e. ad hoc in the
choice of linguistic phenomena), but also for being ad hoc theoretically: CDA has been criticized for being theoretically eclectic. See Jeffries (2000) and the references mentioned there for
a discussion about this characteristic of stylistic research.
Chapter 11. Systematic stylistic analysis 227
2. Method
The analysis of Geert Wilders contribution to the debate on Islamic activism is
based on a method proposed by Leech and Short (2007). So far, their approach
is the best developed method in which a linguistic checklist forms the basis for
stylistic analysis. Although Leech and Short focus on the analysis of literary texts,
their method is, in principle, applicable to non-literary texts as well (Leech and
Short 2007:62). It has three important features. First, the checklist itself, which
involves possible relevant linguistic means for stylistic analysis (Appendix 1). It
consists of four categories:
A.
B.
C.
D.
Lexical categories
Grammatical categories
Figures of speech
Context and cohesion
Each of the four categories lists numerous linguistic phenomena that can be relevant for the stylistic analysis of a particular text. As such, the checklist can be
used as a heuristic tool to find linguistic means bottom up. Because one cannot
tell in advance which factors are relevant and which are not, the checklist can be
helpful in finding these means, without excluding phenomena beforehand.
The other two important features of the Leech and Short method concern
practicing stylistic analysis. The authors point out that it is often useful to make
use of a point of comparison to facilitate the analysis: by contrasting the style of a
text with a very different one it is easier to distinguish relevant linguistic elements
of a text (Leech and Short 2007:4144). For that reason, Wilders speech will be
analysed in comparison to another speech from the same debate: the contribution
by the then Minister of Integration, Ella Vogelaar. Like Wilders, she presents her
ideas on integration during the debate.
Leech and Short also argue that it is essential to make a connection between
micro-level and macro-level phenomena, i.e., the checklist is used to trace at a
micro level various linguistic means that point towards a common literary purpose (the macro level, e.g., the theme of a novel) (Leech and Short 2007:88). Leech
and Short focus solely on the analysis of literary texts, and in this type of texts the
relation between micro and macro level is relatively clear. They do not go into the
macro level of political discourse.
What, then, is the macro level in the case of political speeches held in parliament? It is characteristic for this type of texts that a politician tries to convince
his audience of his political views. Therefore, I would like to suggest that in parliamentary speeches the politicians main viewpoints are expressed at the macro
level. In the case of the debate on Islamic Activism, Wilders and Vogelaar take
very different political positions with respect to integration. Wilders argues that
what he calls the Islamification of the Netherlands has to be stopped; Vogelaar
speaks about integration in general, and argues that integration is a difficult and
long-term process. She pleads that autochthones and immigrants both need to
contribute to the process of integration (cf. Van Leeuwen 2009). In other words,
Vogelaars position is more abstract and more nuanced than Wilders position. It
should be noted that the different political roles of Wilders and Vogelaar influence
their political positions: Minister Vogelaar expresses the position of the government, while Wilders, as a member of the opposition, presents only the ideas of his
own Party for Freedom he can afford to be less compromising in his position.
Moreover, the different political roles also have consequences for how both speakers can formulate their positions and arguments: Wilders can formulate his views
relatively sharply, while Vogelaar has to weigh her words more carefully.
The media respond very differently to political positions that Wilders and
Vogelaar take with regard to integration. Intuitive media reports conclude that
Wilders language use is very clear3, while Vogelaar is criticized for her unclear,
veiled or woolly language use.4 These intuitive judgments about the language
use of both speakers could be interpreted as an indication of how the political
position of both speakers is perceived by the public. Because of the contrast, they
present a good starting point for our stylistic analysis to find out which linguistic means can contribute to the intuitive judgments about Wilders clear and
Vogelaars woolly speeches.
This question forms the basis for the stylistic analysis in the next section, in
which the use of a checklist will be put to the test. In Section 3, various linguistic
means from all four categories of the Leech and Short checklist will be discussed.
These linguistic means were selected during an analysis on every subsection of the
checklist, i.e., they turned out to be the most salient ones to identify which factors
contribute to the impression of clarity and woolliness in the two speeches. I will
focus successively on linguistic means from categories A (lexicon), C (figures of
speech) and D (context and cohesion): categories that are frequently analysed in
3. Wilders clear language is for instance indicated by the fact that he won a plain language
award in 2007, or by the judgment of the Dutch political scientist Andr Krouwel who stated
that Wilders scores low on argumentation, but speaks in very clear phrases (see http://www.
kennislink.nl/web/show?id=275381; accessed on September 8, 2010).
4. The woolliness of Vogelaars language use is for instance indicated by Heerma van Voss
(2008) who states that Vogelaar was designated a communication strategist who occupied himself with minister Vogelaar and her notorious woolly language use.
political discourse. Category B (grammar) is less often used in analysis, and will
be discussed in Section 3.4. Section 4 will reflect on the analysis presented in view
of the limitations and affordances of the checklist for political discourse analysis.
3. Stylistic analysis of Wilders and Vogelaars speeches
In the stylistic analysis presented below, abbreviations in brackets (e.g., A1, C3)
refer to subcategories in the Leech and Short (2007) checklist (see Appendix 1).
Abbreviations used in the quantified data must be read as follows: W = Wilders;
V = Vogelaar; w = number of words. The examples are translated from Dutch by
the author (cf. Appendix 2).
3.1
Lexical categories
Adverbs & adjectives (A3 + A5)

A major stylistic difference between Geert Wilders and Ella Vogelaar is their different use of adverbs and adjectives of quantity, intensity and time. In Vogelaars
speech (Example 1), these adjuncts have a mitigating function significantly more
often than in Wilders contribution (V: 26/1666 w; W: 9/1352 w; G2(1)=5.45,
p<.05):5
(1)
Some habits and traditions get nearly noiselessly accepted in society, but we
also see that less pleasant and sometimes even negative ways of changes cause
friction and tension in society.6
When the adverbs are placed on scales (quantity: everything-much-little-nothing;

intensity: extremely-very-somewhat-to a small extent; time: always-sometimesnever), it is striking that in Wilders speech most of these adverbs denote an endpoint on the semantic scale. Whereas Vogelaar quite frequently uses mitigating
modifiers, Wilders leaves nothing to the imagination (Example 2).
5. The frequencies in both speeches have been calculated by using a log likelihood calculator http://ucrel.lancs.ac.uk/llwizard.html; accessed on September 8, 2010). A loglikelihood test
enables a comparison between frequencies in corpora, even if the investigated phenomena are
relatively rare (Vis et al. 2009:415). For the computation of differences in sentence length (see
Section 3.4), a students t-test was used.
6. All examples that are presented in this chapter were translated from Dutch (authors translations for Vogelaars quotes, for the Wilders speech his own translation was used). All Dutch
original texts are provided in Appendix 2.
(2) The Koran is [] a book which is completely against our legal order and our
democratic institutions. In this light, it is absolutely necessary to ban the Koran
for the defence and reinforcement of our civilization and our constitutional
state. []
In Wilders speech, the adjuncts of quantity, intensity and time seem to be part
of the larger stylistic phenomenon called promotional language (Pander Maat
2007): a phenomenon that contains all linguistic means that can be used to enforce
someones standpoints (Schellens 2006:17). Promotional language can be found in
adjectives and adverbs, but also in other linguistic phenomena, such as substantives, verbs and figurative language. In this chapter, promotional language as such
will not be analysed in detail: that would require a study in itself (cf. Pander Maat
2007; Schellens 2006). However, it is striking that Wilders uses this promotional
language technique frequently (Example 3).

(3) The majority of Dutch citizens have become fully aware of the danger, and
regard Islam as a threat to our culture. [] Many Dutch citizens are fed
up to the back teeth and yearn for action. However, their representatives in
The Hague are doing precisely nothing. They are held back by fear, political
correctness or simply electoral motives.
Abstract nouns (A2)

The speeches also differ in the use of abstract nouns: nouns denoting events, situations, processes, psychological or social phenomena, e.g., words like polarization,
equality, issues, communities, religion, change of mentality, feelings, emancipation,
etc. Abstract nouns also include empty signifiers (cf. Montesano Montessori,
this volume). The differences in the use of abstract nouns are significant for both
tokens (V: 228/1666 w; W: 126/1352 w; G2(1) = 12.36, p < .001) and types (G2(1)=
4.83, p < .05). The higher level of abstractness of Vogelaars speech can partly be
explained by the content of her contribution: she speaks about integration-ingeneral, while Wilders focuses more concretely on the integration of one specific
group in Dutch society: Muslims (see Section 2).7
7. There are several factors that can explain the stylistic differences between Wilders and
Vogelaar. Two have already been mentioned: the different political standpoint, and the different political role (cf. Section2). Additional factors could be preparation (Wilders had written
his speech beforehand; Vogelaar had not (cf. Vogelaar and Bosma 2009), and gender (see for
instance Oversteegen and Missioura 2009). Although such differences can indeed partly explain
the differences mentioned, they do not alter the fact that the described differences between both
speakers exist, and that both speakers construe their ideas about integration stylistically in a very
different way.
General: Definite vs. indefinite articles (A1)

A final relevant lexical aspect is the use of definite and indefinite articles (A1).
Vogelaar often refers to groups of people or to concepts in a generic, indefinite
way (when you talk to people, you get to know each other / Of course, faith
and religion are part of integration issues). Wilders instead, prefers to use the
definite article de Islam (lit. the Islam), the Dutch people, the politicians in
The Hague, etc.: clear-cut entities and concepts that are presented as unities. By
doing so, Wilders abstracts from the diversity within groups or concepts that can
be found in reality; the simplifications contribute to the clarity of his message
(Van Leeuwen 2009).
3.2
Figures of speech
Metaphor (C3)
Unlike Vogelaar, Wilders makes use of consistent imagery to present his ideas.
He systematically speaks about the Islamification in terms of war (Example 4),

(4) Madam speaker, approximately 1400 years ago war was declared on us by an
ideology of hate and violence [].
[] the Islamic incursion must be stopped. Islam is the Trojan Horse in Europe.
She [minister Vogelaar] is betraying Dutch culture [].
Islam aims to dominate, subject, kill and wage war.
The war metaphor contributes to the simplification of Wilders message, and with
that to the clarity of his speech: the war metaphor enables him to create clear
distinctions between good and evil, between aggressor (Islam), victims (millions
of Dutch people), cowards (the Dutch government) and defenders of freedom
(Wilders and his Party for Freedom). In addition, the suggested war situation can
function as a justification for Wilders radical viewpoints: in times of peace, his
views would be far less self-evident (cf. Lakoff and Johnson 1980; Lammerts and
Verhagen 1994).
Anaphora, parallelism (C1)

The impression of clarity of Wilders speech is further supported by frequent use
of parallelism: sentences or parts of sentences with similar grammatical structures.
For instance, Wilders ends his speech as in Example (5):

(5) Minister [Balkenende], on behalf of a great many Dutch citizens: stop the
Islamification of the Netherlands! Mr Balkenende, a historic task rests on your
shoulders. Be courageous. Do what many Dutch citizens are screaming out
for. Do what the country needs. Stop all immigration from Muslim countries,
ban the building of new mosques, close all Islamic schools, ban burkahs and
the Koran. Expel all criminal Muslims from the country, including those
Moroccan street terrorists that drive people mad. Accept your responsibility!
Stop Islamification!Enough is enough, Mr Balkenende. Enough is enough.
Vogelaar does not use rhetorical figures that have a structuring function: sentences
which are composed in a parallel way are much less frequent in her speech.
3.3
Cohesion
Referring expressions (D1)

Referring expressions are another important feature that adds to the impression
of clarity and woolliness of the two speeches. For a clear text it is important
that referring expressions are not ambiguous or difficult to resolve (cf. Burger and
de Jong 2009; Sanders and Spooren 2007). In Dutch, demonstratives (this, that,
those) and pronominal adverbs (therefore, therein, hereto) are two frequently used
types of referring expressions. Wilders and Vogelaar differ significantly in their use
(V: 62/1666 w; W: 32/1352 w; G2(1)=4.50; p<.05).
However, the way in which both speakers use these referring expressions is
more important than their absolute frequency. In Vogelaars speech, the referring
expressions contribute to the woolliness of her speech. In 18 cases (29%) it is not
quite clear what the speaker is referring to. The numbered/italicized examples in
Example (6) are representative for this aspect of her speech.

(6) It [the policy] is about stimulating womens liberation, about strengthening

those communities defences against radicalization and about making honour
related violence debatable. In these communities themselves it has to become
clear what is acceptable, and what isnt. [1] Taboos around these kinds of
subjects have to be broken within these communities. In the integration policy
we start from the preservation of achievements in our society, like equality of
men and women regardless of their sexual inclination or religion. It has taken
our society long enough to come to broad acceptance of [2] these equalities. I
will promote and defend [3] these forcefully. But I also say that [4] this means
that we need realism and patience.
These kinds of subjects [1] and these equalities [2] are difficult to process
because it is not immediately clear what they refer to precisely. These [3] is relatively difficult because it does not refer to these qualities in the preceding sentence, but further back, to equality of men (). A similar thing holds for this
[4]: it does not refer to the same concepts as these in the preceding sentence, but
to the fact that it has taken society long enough to come to a broad acceptance.
Regarding referring expressions, Wilders contribution is much clearer than
Vogelaars speech: his demonstratives and pronominal adverbs are easy to solve
in all cases (Example 7).

(7) Apart from conquest, Madam Speaker, Islam is also bent on installing a totally
different form of law and order, namely Sharia law. This makes Islam, apart
from a religion for hundreds of millions of Muslims also, and in particular, a
political ideology.
3.4
Grammatical categories
While checking every subsection of the Leech and Short (2007) checklist, the linguistic elements discussed so far turned out to be the most salient aspects contributing to the impression of clarity and woolliness. In this final subsection, the
less frequently analysed category of grammatical phenomena (category B) will be
elaborated on and applied. It will be shown that it is important for stylistic analysis
to take this category into account as well.
Sentence length and sentence complexity (B2)

Based on the intuitive judgments, it can be expected that the sentences in Wilders
and Vogelaars speeches differ in length as well as complexity (B2). This intuition
indeed turns out to be correct. Wilders average sentence length is 15.2 versus
19.8 for Vogelaar, a significant difference (t (171) = 2.62, p < .01). Regarding the
complexity of sentences, it is striking that more than 58% of Wilders sentences do
not contain subordinate clauses (52 out of 89 sentences), versus 31% in the case of
Vogelaars speech (26 out of 84 sentences). This difference in sentence structure is
also reflected in the amount of finite subordinated clauses (B3) that both speakers
use: 43 vs. 87. Differentiated among different types of finite subordinated clauses,
the result is the more detailed overview in Table 1.
Table 1. Finite subordinated clauses in the speeches by Wilders and Vogelaar.
Type of finite subordinate clause
Wilders
Vogelaar
Adverbial clauses
Non-restrictive relatives
Restrictive relatives
Complementation
Total
8
11
12
12
43
13
9
21
44
87
The main difference in the use of finite subordinate clauses can be found in the complementation category, which is used 12 times by Wilders, and 44 times by Vogelaar.
These results made it interesting to also analyse complementation in more detail.
Clause types: Complementation (B3)

Complementation constructions consist of a matrix- and complement- clause,
in which the complement-clause gives a description of reality, while the matrixclause rather gives a description of the speakers stance towards that description of
reality as is illustrated by the following example (Verhagen 2005).
MATRIX-CLAUSE

a. The director of GenTech
expects that

b. Others believe that

c. but nobody doubts that

d. The question is whether
or whether

COMPLEMENT-CLAUSE
clones of mammalian embryosit will become
possible in the near future.
it may take somewhat longer
the cloning of a full-grown sheep or horse will
be a reality within ten years.
society is mentally and morally ready for this
we will once again be hopelessly overtaken by
the technical developments.
(Verhagen 2005:96)
In each sentence, the matrix clause invites the reader to adopt a stance towards a
description of reality, which is given in the complement clause. The expression of
stance in the matrix clause can be explicitly related to the person whose stance is
represented: in sentence (a) this is the directors viewpoint, and in (b) and (c) the
standpoint of others and nobody. In (d) however, an impersonal complementation
construction can be observed: the matrix clause denotes a cognitive stance which
is not explicitly related to anybody in particular. In such cases the context gives a
decisive indication of whose stance is adopted (Verhagen 2005:131137): e.g., in
sentence (d), the matrix-clause expresses the perspective of the writer.
Verhagen argues that making use of complementation constructions can
cause certain rhetorical effects. He gives the following example (Verhagen
2005:105107):

a.

b.

c.
Will we be in time for the football match?

It was scheduled for 4 p.m.
I think it was scheduled for 4 p.m.
Michael said that it was scheduled for 4 p.m.
The argumentative orientation of each answer is the same: in the same context
(say, it is 2 p.m., and we are close to the stadium), each of the three responses
guides the addressee to draw the same conclusion (probably yes, we will be in
time). However, the argumentative strength of each answer varies: sentence (a)
presents the relevant information directly, as a matter of fact. In sentence (b),
this information is explicitly related to the subjective perspective of the speaker.
As a result, the possibility is implied that there is a difference between the point
of view taken and reality. In other words, by explicitly presenting his perspective
on the issue, the speaker evokes the idea that other perspectives are also possible.
As a consequence, the sentence (b) leaves more room for negotiation and discussion than does sentence (a): the argumentative strength is less. The argumentative
strength of sentence (c) is even weaker than the (b), because here the possibility
exists that the speaker of the utterance (I) and Michael have a different point of
view about the question whether they will be in time.8
How are complementation constructions used in the speeches by Wilders and
Vogelaar? In Table 2, the complementation constructions are divided into different
types (based on the grammatical subject in the matrix clause) and linked to whose
perspective is adopted.
Table 2. Complementation constructions in the speeches by Wilders and Vogelaar from
a formal and a functional perspective.
Whos perspective?
Type of complementation
Speakers perspective
1st person singular
14
1st person plural
10
Impersonal
16
2nd person singular
2nd person plural
10
Others perspective
3rd person singular

3rd person plural
Impersonal
Total
Wilders
Vogelaar
12
44
First of all, Table 2 shows that Vogelaars speech contains 40 cases of complementation constructions in which the speakers perspective is expressed in the matrix
clause (Examples 811).
8. There are contexts conceivable in which utterance B has less argumentative strength than
utterance C, e.g. in a context in which Michael is an expert on the topic that is under discussion.
However, in such a context it still holds that both the C- and B-utterances have less argumentative strength than the A-sentence, i.e. the answer with a complementation construction has less
argumentative strength than an answer without such a construction. It is this difference that
occupies centre stage in the analysis of both speeches.
(8)
I believe that my role as minister of integration is to raise these matters within
the communities in which they occur.
(9)
More and more often, we see that Muslims are being equated with extremists
and enemies of democracy.
(10) It is a fact that the acceptance of this religion is complicated through [].
(11) For the authorities, this means that religion as such is a collateral factor which
has to be taken into account in our policy [].
In Wilders speech, only one construction of this type can be observed in Example
(12).
(12) Madam Speaker, let us ensure that the third Islamic invasion, which is currently
in full spate, will be stopped ().
Typical for Wilders speech is the lack of complementation as in Examples (1315).

(13) Madam Speaker, the Islamic incursion must be stopped. Islam is the Trojan
Horse in Europe. If we do not stop Islamification now, Eurabia and Netherabia
will just be a matter of time.
(14) Very many Dutch citizens, Madam Speaker, experience the presence of
Islam around them. They have had enough of burkas, headscarves, the ritual
slaughter of animals, so-called honour revenge, blaring minarets, () and
the enormous overrepresentation of Muslims in the area of crime, such as
Moroccan street terrorists.
(15) Madam Speaker, the Koran is a book that incites to violence and the distribution of such texts is unlawful according to Article 132 of our Penal Code. In
addition, the Koran incites to hatred and calls for murder and mayhem. The
distribution of such texts is a criminal offence by Article 137(e).
What is the rhetorical effect of this difference in the use of complementation constructions? It is striking that Vogelaar frequently describes her viewpoints as her
perspective on integration, while Wilders presents his ideas primarily as facts. As
a result, Vogelaar leaves room for discussion and negotiation, whereas Wilders
leaves minimal room for that: the lack of complementation constructions in
Wilders language use contributes to the certainty with which he presents his ideas.
This is further supported by another striking difference shown in Table 2,
that Wilders does use complementation constructions more than once, but only
to present the ideas of other people (Example 16).
(16) Minister Donner said earlier that he could imagine that Sharia law could be
introduced in the Netherlands only if a majority would support it.
In other words, Wilders presents the ideas of others as perspectives whereas his
own ideas are presented as facts. This factuality and certainty by which Wilders
presents his own ideas contributes to the clarity of his message: he leaves minimal
room for alternative views.
By presenting opinions as facts, it looks as if Wilders is being objective.
However, his message is extremely subjective, as can be seen in for instance his
use of promotional language (cf. Section 4.1) and his presentation of what he calls
the Islamification in terms of war (cf. Section 4.2).
4. Conclusion
This chapter presents a detailed stylistic analysis of the speeches by the Dutch
politicians Geert Wilders and Ella Vogelaar in the debate on Islamic activism
in the Dutch Lower Chamber (2007). A checklist (Leech and Short 2007) was
used to find linguistic means that could contribute to intuitive judgments about
the language use of both politicians (clear vs. woolly language), which form an
indication of the different perceptions of Wilders and Vogelaars political position
with regard to integration. The aim of the analysis was to test the applicability of a
checklist in analysing stylistic choices in political discourse.
A point of critique that has been raised against stylistic analysis of political
discourse is that analyses are often ad hoc: they fail to clarify why some linguistic
means are analysed, and others are not. Although a checklist cannot really solve
this problem, this does not mean that a checklist is not useful in stylistic analysis,
on the contrary. When applying the checklist, the analysis is still ad hoc in the sense
that the analyser will always make selective use (Leech and Short 2007:66) of the
checklist. That is, quite some categories of the checklist turn out to yield no stylistic differences after making a first inventory of the data. These categories are not
further analysed. When other categories do show stylistic differences it is possible
that some of these categories are eventually excluded as well, if they appear to be
irrelevant to the analysts research question. In other words, the choice for analyzing
some categories from the checklist and not others is the result of a combination of
bottom-up analysis (searching for relevant stylistic means using the checklist) and
of top-down analysis that is guided by the research question (e.g., which stylistic
characteristics can contribute to the intuitive judgments about Wilders clear and
Vogelaars woolly contribution to the debate?). This means that the selection process is not completely transparent. However, the use of a checklist is an improvement: the reader gets at least some idea of factors the analyser has selected as being
relevant, and which are ignored in the bottom-up analysis.
Another advantage of the checklist is its heuristic function: the checklist can
help to find relevant linguistic means that could easily be overlooked otherwise.
For instance, in the present analysis the category of referring expressions is not
immediately obvious, nevertheless it turns out to be important. Moreover, the
checklist helps to focus on grammatical phenomena in stylistic analysis: a category to which relatively little attention has been paid in the analysis of political
discourse. The checklist makes explicit that style is a combination of linguistic
means that can be found at all layers of a text, including grammar.
A further aspect of using a checklist is that it presupposes a lot of linguistic background: behind every category mentioned, a whole world of linguistics
is hidden. In theory, every category of the checklist could be a study in itself.
This may hamper the use of a checklist, because the categories mentioned in the
checklist are often not directly applicable to text analysis. In practice, a category
often has to be split up in more detail before interpretation can reveal rhetorical
effects (see for instance the analysis of finite subordinated clauses / complementation constructions in Section 3.4). However, this aspect is a problem of stylistic
research as such: without the use of a checklist, the problem would also exist.
In conclusion, the use of a linguistic checklist does not provide a panacea for
stylistic analysis: it does not reduce the analysis of style to a relatively uncomplicated activity. However, a linguistic checklist is a good starting point for stylistic
analysis: it helps the analyser to find linguistic categories that are relevant for his
research question, and gives the reader an idea of which factors the analyser has
taken into account (and which have been left out), thus making the analysis more
transparent. As such, it is a drawback that the few proposals for working with
a checklist have not been followed up by others so far, as the use of a linguistic checklist provides a valuable contribution to the stylistic analysis of political
discourse.
References
Burger, P. and J. de Jong. 2009. Handboek Stijl: Adviezen voor Aantrekkelijk Schrijven. Groningen:
Noordhoff Uitgevers.
Corbett, E.P.J. 1990. Classical Rhetoric for the Modern Student. 3rd ed. Oxford: Oxford University Press.
Fairclough, N. 1992. Discourse and Social Change. Cambridge: Polity Press.
Fairclough, N. 2001. Language and Power. London and New York: Longman.
Fahnestock, J. 2009. Quid pro nobis, Rhetorical stylistics for argument analysis. In F.H. v. Eemeren
(ed.), Examining Argumentation in Context: Fifteen Studies on Strategic Maneuvering. Amsterdam/Philadelphia: John Benjamins, pp. 191220.
Foss, S K. 2004. Rhetorical Criticism: Exploration and Practice. 4th ed. Long Grove, Illinois:
Waveland Press.
Fowler, R. 1991. Language in the News: Discourse and Ideology in the Press. London: Routledge.
Fowler, R. and G. Kress. 1979. Critical linguistics. In R. Fowler, B. Hodge, G. Kress & T. Trew
(eds.), Language and Control. London: Routledge & Kegan Paul.
Halliday, M.A.K. and C.M.I.M. Matthiessen. 2004. An Introduction to Functional Grammar.
3rded. London: Arnold.
Heerma van Voss, S. 2008. Nu gaf Vogelaar wl antwoord; Minister krijgt forse kritiek van kamer
om wijkenaanpak. Nrc.next, April 24.
Jeffries, L. 2000. Dont throw out the baby with the bathwater: in defense of theoretical eclecticism in stylistics. PALA Occasional Papers 12.
Lakoff, G. and M. Johnson. 1980. Metaphors we Live By. Chicago: University of Chicago Press.
Lammerts, A. and A. Verhagen. 1994. De oorlog in de krant. Perspectieven in Taalbeheersingsonderzoek, Dordrecht: ICG Publications. pp. 375-384.
Leech, G.N. and M.H. Short. 2007. Style in Fiction, A Linguistic Introduction to English Fictional
Prose, 2nd ed. London: Longman.
Oversteegen, E. and O. Missioura. 2009. Taal en gender in de advocatuur. Een stilistische analyse
van pleidooien van vrouwelijke en mannelijke advocaten. Neerlandstiek.nl. [http://www.
neerlandistiek.nl/09.05/]
Pander Maat, H. 2007. How promotional language in press releases Is dealt with by journalists:
Genre Mixing of Genre Conflict? Journal of Business Communication 44, pp. 5995. DOI:
10.1177/0021943606295780
Poole, B. 2010. Commitment and criticality: Faircloughs Critical Discourse Analysis. Journal of
Applied Linguistics 20 (2), pp. 137155. DOI: 10.1111/j.1473-4192.2009.00234.x
Sanders, T. and W. Spooren. 2007. Discourse and text structure. In D. Geeraerts and H. Cuyckens (eds.) The Oxford Handbook of Cognitive Linguistics. Oxford: Oxford University Press,
pp. 916941.
Schellens, P.J. 2006. Bij vlagen loepzuiver, Over argumentatie en stijl in betogende teksten.
Oration, Radboud University. Nijmegen.
Van Leeuwen, M. 2009. Het hoofdzinnenbeleid van Wilders, Over de stijl van Geert Wilders en
Ella Vogelaar. Tekstblad 15 (2), pp. 611.
Van Rees, M.A. 2005. Critical Discourse Analysis en retorische analyse. Tijdschrift voor Taalbeheersing 25 (1), pp. 8797.
Verhagen, A. 2005. Constructions of Intersubjectivity, Discourse, Syntax and Cognition. Oxford:
Oxford University Press.
Vis, K., J. Sanders and W. Spooren. 2009. Subjectiviteit door de jaren heen: Conversationalisatie
in journalistieke teksten. In W. Spooren, M. Onrust and J. Sanders (eds.), Studies in Taalbeheersing 3. Van Gorcum, pp. 405418.
Vogelaar, E. and O. Bosma. 2009. Twintig Maanden Knettergek: Dagboek van een Ministerschap.
Uitgeverij Balans.
Wodak, R. 2006. Critical linguistics and critical discourse analysis. In J.-O. stman and
J. Verschueren(eds.), Handbook of Pragmatics. Amsterdam/Philadelphia: John Benjamins.
Wodak, R. and R. de Cillia. 2006. Politics and language: Overview. In K. Brown (ed.), Encyclopedia of Language & Linguistics. Oxford: Elsevier, pp. 707719.
Appendix 1
A checklist of linguistic and stylistic categories (Leech and Short 2007)
A: Lexical categories
1. GENERAL. Is the vocabulary simple or complex? Formal or colloquial? Descriptive or
evaluative? General or specific? How far does the writer make use of the emotive and other
associations of words, as opposed to their referential meaning? Does the text contain idiomatic
phrases or notable collocations, and if so, with what kind of dialect or register are these idioms
or collocations associated? Is there any use of rare or specialized vocabulary? Are any particular
morphological categories noteworthy (e.g. compound words, words with particular suffixes)?
To what semantic fields do words belong?
2. NOUNS. Are the nouns abstract or concrete? What kinds of abstract nouns occur (e.g.
nouns referring to events, perceptions, processes, moral qualities, social qualities)? What use is
made of proper names? Collective nouns?
3. ADJECTIVES. Are the adjectives frequent? To what kinds of attribute do adjectives refer?
Physical? Psychological? Visual? Auditory? Colour? Referential? Emotive? Evaluative? etc. Are
adjectives restrictive or non-restrictive? Gradable or non-gradable? Attributive or predicative?
4. VERBS. Do the verbs carry an important part of the meaning? Are they stative (referring
to states) or dynamic (referring to actions, events, etc.)? Do they refer to movements, physical
acts, speech acts, psychological states or activities, perceptions, etc.? Are they transitive, intransitive, linking (intensive), etc.? Are they factive or non-factive?
5. ADVERBS. Are adverbs frequent? What semantic functions do they perform (manner,
place, direction, time, degree, etc.)? Is there any significant use of sentence adverbs (conjuncts
such as so, therefore, however; disjuncts such as certainly, obviously, frankly)?
B: Grammatical categories
1. SENTENCE TYPES. Does the author use only statements (declarative sentences), or do
questions, commands, exclamations or minor sentence types (such as sentences with no verb)
also occur in the text? If these other types appear, what is their function?
2. SENTENCE COMPLEXITY. Do sentences on the whole have a simple or complex structure? What is the average sentence length (in number of words)? What is the ratio of dependent to independent clauses? Does complexity vary strikingly from one sentence to another? Is
complexity mainly due to (i) coordination, (ii) subordination, or (iii) parataxis (juxtaposition
of clauses or other equivalent structures)? In what parts of a sentence does complexity tend to
occur? For instance, is there any notable occurrence of anticipatory structure (e.g. of complex
subjects preceding the verbs, of dependent clauses preceding the subject of a main clause)?
3. CLAUSE TYPES. What types of dependent clause are favoured: relative clauses, adverbial
clauses, different types of nominal clauses (that-clauses, wh-clauses, etc.)? Are reduced or nonfinite clauses commonly used and, if so, of what type are they (infinitive clauses, -ing-clauses,
-ed clauses, verbless clauses)?
4. CLAUSE STRUCTURE. Is there anything significant about clause elements (e.g. frequency
of objects, complements, adverbials; of transitive or intransitive verb constructions)? Are there
any unusual orderings (initial adverbials, fronting of object of complement, etc.)? Do special
kinds of clause construction occur (such as those with preparatory it or there)?
5. NOUN PHRASES. Are they relatively simple or complex? Where does the complexity lie
(in premodification by adjectives, nouns, etc., or in postmodification by prepositional phrases,
relative clauses, etc.)? Note occurrence of listings (e.g. sequences of adjectives), coordination or
apposition.
6. VERB PHRASES. Are there any significant departures from the use of the simple past
tense? For example, notice occurrences and functions of the present tense; of the progressive
aspect (e.g. was lying); of the perfective aspect (e.g. has/had appeared); of modal auxiliaries (e.g.
can, must, would, etc.) Look out for phrasal verbs and how they are used.
7. OTHER PHRASE TYPES. Is there anything to be said about other phrase types: prepositional phrases, adverb phrases, adjective phrases?
8. WORD CLASSES. Having already considered major or lexical word classes, we may here
consider minor word classes (function words): prepositions, conjunctions, pronouns, determiners, auxiliaries, interjections. Are particular words of these types used for particular effect
(e.g. the definite or indefinite article; first person pronouns I, we, etc.; demonstratives such as
this and that; negative words such as not, nothing, no)?
9. GENERAL. Note here whether any general types of grammatical construction are used to
special effect; e.g. comparative or superlative constructions; coordinative or listing constructions; parenthetical constructions; appended or interpolated structures such as occur in casual
speech. Do lists and coordinations (e.g. lists of nouns) tend to occur with two, three or more
than three members? Do the coordinations, unlike the standard construction with one conjunction (sun, moon and stars), tend to omit conjunctions (sun, moon, stars) or have more than one
conjunction (sun and moon and stars)?
C: Figures of speech, etc.
Here we consider the incidence of features which are foregrounded by virtue of departing in
some way from general norms of communication by means of the language code; for example,
exploitation code. For identifying such features, the traditional figures of speech (schemes and
tropes) are often useful categories.
1. GRAMMATICAL AND LEXICAL. Are there any cases of formal and structural repetition
(anaphora, parallelism, etc.) or of mirror-image patterns (chiasmus)? Is the rhetorical effect of
these one of antithesis, reinforcement, climax, anticlimax, etc.?
2. PHONOLOGICAL SCHEMES. Are there any phonological patterns of thyme, alliteration,
assonance, etc.? Are there any salient rhythmical patterns? Do vowel and consonant sounds pattern or cluster in particular ways? How do these phonological features interact with meaning?
3. TROPES. Are there any obvious violations of, or departures from, the linguistic code? For
example, are there any neologisms (such as Americanly)? Deviant lexical collocations (such
as portentous infants)? Semantic, syntactic, phonological, or graphological deviations? Such
deviations (although they can occur in everyday speech and writing) will often be the clue to
special interpretations associated with traditional poetic figures of speech such as metaphor,
metonymy, synecdoche, paradox and irony. If such tropes occur, what kind of special interpretation is involved (e.g. metaphors can be classified as personifying animising, concretising,
synaesthetic, etc.)? Because of its close connection with metaphor, simile may also be considered
here. Does the text contain any similes, or similar constructions (e.g. as if constructions)? What
dissimilar semantic fields are related through simile?
D: Context and cohesion
Cohesion: ways in which one part of a text is linked to another (the internal organisation
of the text).
Context: the external relations of a text or a part of a text, seeing it as a discourse presupposing a social relation between its participants (author and reader; character and character,
etc.), and a sharing by participants of knowledge and assumptions.
1. COHESION. Does the text contain logical or other links between sentences (e.g. coordinating conjunctions, or linking adverbials)? Or does it tend to rely on implicit connections
of meaning? What sort of use is made of cross-reference by pronouns (she, it, they, etc.)? By
substitute forms (do, so, etc.), or ellipsis? Alternatively, is any use made of elegant variation
the avoidance of repetition by the substitution of a descriptive phrase (as, for example, the old
lawyer or her uncle may substitute for the repetition of an earlier Mr Jones)?

Are meaning connections reinforced by repetition of words and phrases, or by repeatedly
using words from the same semantic field?
2. CONTEXT. Does the writer address the reader directly, or through the words or thoughts
of some fictional character? What linguistic clues (e.g. first person pronouns I, me, my, mine)
are there of the addresser-addressee subject? If a characters words or thoughts are represented,
is this done by direct quotation (direct speech), or by some other method (e.g. indirect speech)?
Are there significant changes of style according to who is supposedly speaking or thinking the
words on the page?
Appendix 2
Translations of quotes from speeches by Vogelaar (V) (authors translation) and Wilders (W)
(Wilders translation).
(1) V: Sommige gebruiken en tradities worden vrijwel geruisloos in de samenleving aanvaard,
maar wij zien ook dat minder prettige en soms zelfs negatieve kanten van veranderingen
wrijving en spanningen in de samenleving veroorzaken.
(2) W: De Koran is levensgevaarlijk en volledig in strijd met onze rechtsorde en democratische
rechtsstaat. Het zal de rechtsstaat, de vrijheid van godsdienst en onze Westerse beschaving
alleen maar versterken als wij de Koran verbieden. Ter verdediging en versterking van onze
rechtsstaat en beschaving is het dan ook bittere noodzaak de Koran te verbieden.
(3) W: () De meerderheid van de Nederlanders is namelijk doordrongen van het feit dat de
islam een gevaar is. () veel Nederlanders zijn het spuugzat en hunkeren naar actie. Maar
de Haagse politiek doet helemaal niets, tegengehouden door angst, politieke correctheid
of simpelweg electorale motieven.
(4) W: Mevrouw de voorzitter, ongeveer 1400 jaar geleden is ons de oorlog verklaard door een
ideologie van haat en geweld ().

() Die toenemende islamisering moet worden gestopt. De Islam is het paard van Troje
in Europa.

Zij toont () aan dat zij de Nederlandse cultuur verraadt.

De islam wil overheersen, onderwerpen, doden en oorlog voeren.
(5) W: Ik doe () [een] persoonlijke beroep op de minister-president namens heel veel
Nederlanders. Stop de islamisering van Nederland. Er rust een historische taak op uw
schouders, mijnheer Balkenende. Wees moedig. Doe waar veel Nederlanders om schreeuwen. Doe wat Nederland nodig heeft. Stop de immigratie uit moslimlanden. Sta geen
enkele nieuwe moskee meer toe. Sluit de islamitische scholen. Verbied de boerka. Verbied
de Koran. Zet criminele moslims, zoals die Marokkaanse straatterroristen waar mensen in
het land echtknettergek van worden, nu een keer het land uit. Neem uw verantwoordelijkheid. Stop de islamisering. Genoeg is genoeg, mijnheer Balkenende. Genoeg is genoeg.
(6) V: Het gaat om het bevorderen van de emancipatie van vrouwen, het versterken van de
weerbaarheid van die gemeenschappen tegen radicalisering en het bespreekbaar maken
van eergerelateerd geweld. In die gemeenschappen zelf moet duidelijk worden wat wel en
niet kan. Taboes rond dit sort onderwerpen moeten binnen die gemeenschappen worden
doorbroken. In het integratiebeleid gaan wij uit van het behoud van verworvenheden in
onze samenleving, zoals de gelijkheid van mannen en vrouwen, ongeacht hun seksuele
geaardheid of religie. Het heeft in onze samenleving lang genoeg geduurd om tot een brede
maatschappelijke acceptatie te komen van deze gelijkheden. Ik zal deze met kracht bevorderen en verdedigen. Maar ik zeg er ook bij dat dit betekent dat wij realisme en geduld
nodig hebben.
(7) W: Naast verovering is de islam ook uit op het instellen van een totaal andere maatschappelijke orde en rechtssysteem, de sharia. Daarmee is de islam behalve een religie voor vele
honderden miljoenen moslims eigenlijk ook een politieke ideologie.
(8) V: Ik denk dat mijn rol als minister voor integratie is om dit soort zaken aan de orde te
stellen binnen de gemeenschappen waarin zij voorkomen.
(9) V: Steeds vaker zien wij dat moslims vereenzelvigd worden met extremisten en vijanden
van de democratie.
(10) V: Het is een gegeven dat de acceptatie van deze religie wordt bemoeilijkt doordat de islam
een aantal sterke uitingsvormen heeft in het publieke domein.
(11) V: Voor de overheid betekent dit dat religie als zodanig een omgevingsfactor is, waarmee
wij in ons beleid rekening moeten houden, maar waarmee wij ons niet inhoudelijk moeten
inlaten.
(12) W: Laten wij ervoor zorgen dat de derde islamitische invasie die nu volop gaande is, tot
stilstand wordt gebracht.
(13) W: Voorzitter, de toenemende islamisering moet worden gestopt. De islam is het paard van
Troje in Europa. Als we de islamisering niet stoppen zijn Eurabi en Nederabi slechts een
kwestie van tijd.
(14) W: Heel veel Nederlanders, meneer de minister-president, zien de islamisering van
Nederland echter elke dag, en heel veel mensen hebben genoeg van boerkas, van hoofddoekjes, van het ritueel slachten van dieren, van eerwraak, van schallende minaretten
() en van de enorme oververtegenwoordiging van moslims in de misdaad, zoals de
Marokkaanse straatterroristen.
(15) W: Voorzitter, de Koran is een opruiend boek en het verspreiden van een opruiend
geschrift is op grond van artikel 132 van ons Wetboek van Strafrecht verboden. Daarnaast
zet de Koran aan tot haat en roept het op tot moord en doodslag; verspreiding van dergelijke teksten is door artikel 137e strafbaar gesteld.
(16) W: Minister Donner zei eerder dat hij de invoering van een sharia in Nederland zich kon
voorstellen, als de meerderheid dat maar zou willen.
chapter 12
Participation and recontextualisation

innewmedia
Political discourse analysis and YouTube
Michael S. Boyd
Universit Roma Tre
The chapter explores the ways in which viewer comments are exploited by the
YouTube community to interact with political discourse. The corpus-based
study, based on three speeches by Barack Obama, aims to uncover the linguistic
means adopted in text comments for positive and negative presentation and for
recontextualisation of the speeches. In line with Critical Discourse Analysis, the
work is premised on the assumption that the speeches rebroadcast on YouTube
reshape linguistic and social practice by providing a wider reception and more
direct access to institutionalized political discourse. Specifically, the work
explores how interaction in the form of text comments influences the medium
of YouTube, the discourse community and the genre of political speech. On a
theoretical level the work considers the important issues of online identity and
participation in the public sphere, with a focus on the various ways recontextualisation is exploited by commenters.
1. Introduction
In a recent overview of Critical Discourse Analysis (CDA), Wodak and Meyer
(2009:11) stress the importance of studying new phenomena, such as participation and new media in Western political systems. New technologies, in fact, are
being combined in innovative ways to forge multi-modal environments that can be
exploited for political advertising (Reisigl and Wodak 2009), on the one hand, and,
for (inter)action to be played out among text consumers, on the other (cf. Gryc
and Moilanen, this volume). YouTube has recently become an important vehicle
in politics, as demonstrated by the wide use of this medium by the candidates,
mass media and the electorate in the 2008 US presidential elections to advertise,
propagate, broadcast, discuss, legitimize, delegitimize, and/or criticize political
actors, positions and texts. Its significance in the 2008 campaign is attested by
246 Michael S. Boyd
the fact that all 16 presidential candidates had their own YouTube channel; seven
presidential candidates announced their candidacies on YouTube; and advocacy
groups and ordinary citizens employed the medium to distribute and exchange
political opinion and messages (Grove 2008). A wide range of traditional political
genres, such as campaign speeches, announcements, debates, campaign advertisements and infomercials, as well as video mash-ups and responses were published
on YouTube, providing a quick and easy way for a wider and more technologized
voting public to access an ever increasing number of campaign texts, and, more
importantly, in their entirety.
Barack Obamas campaign team was particularly effective in applying new
media for political purposes, espousing many of these technologies for fundraising, distributing campaign messages and propaganda, and consensus building.
This strategy helped to reach those voters who had stopped relying on traditional
media sources, such as television and radio, for news and information (Nagourney
2008). Obamas official site Organizing for America (www.barackobama.com),
centred around his underlying campaign message of hope and change, was
expanded to include the stories of common people, a strategy that was also used
(and is still used) on social network sites and in personal emails. Most of Obamas
speeches, announcements and advertisements were immediately uploaded to
YouTube, often in cleaner, tailor-made versions (Heffernan 2009). By March 2008,
one of the most crucial moments in the primary campaign, the Obama team was
uploading 23 videos per day (Grove 2008). One of Obamas speeches from this
period, and undoubtedly one of the most memorable from the entire campaign,
was A More Perfect Union, a 37-minute speech on race. The video continues to
be popular on YouTube with well over 9 million views.1 Throughout the campaign the Obama team continued to post and (re)distribute campaign speeches,
announcements and debates, a practice which continued into his presidency
(Heffernan 2009) and continues to generate public responses.
This chapter explores the phenomena of text comments, community and identity on YouTube in relation to political videos. Specifically, it focuses on the use
of text comments in the various YouTube versions of three popular speeches by
Barack Obama: A More Perfect Union (March 2008), Election Victory Speech
(November 2008) and Inaugural Address (January 2009). The text comments
provide the empirical data for a corpus-based analysis. These three speeches were
selected because they represent distinct sub-genres within the larger genre of
political speeches, and, consequently, it was expected that they would generate
1. As of February 2014. The number refers to the total number views of the two most popular
versions of the video with the entire speech. There are many other shorter versions available.
Chapter 12. Participation and recontextualisation innewmedia 247
different types of comments. Moreover, the speeches were all extremely popular on
YouTube, each with millions of views and, for the most part, a significant number
of text comments. Furthermore, all of the speeches were closely followed by both
the public and the mass media.
The study addresses four main research questions. First, it considers how
the YouTube medium influences the reception conditions of the political speech,
also with reference to a parallel study about the genre of the political speech on
YouTube (Boyd 2011). Second, since the texts in the study are representative of
three distinct subgenres of political speeches, it was postulated that they would
invoke different degrees and forms of participation, and, consequently, different
types of discourse and social practices. Third, the study looks into the role that
individual users and the YouTube community play in (re)shaping social (and
discourse) practices, in line with the premises of the CDA approach. Finally, it
considers the elements of power and domination as mediated through recontextualisation (or, more generally, language use), asking, specifically, if the
(re)broadcasting of political speeches via YouTube encourages public-space dialogue, participation and democratization in the political sphere.
From a theoretical perspective, the work is premised on the assumption that
the political speeches rebroadcast on YouTube are an important force in reshaping social practice, providing a wider reception and more direct access to the
genre of political speech (and, of course, the texts themselves2). The study explores
how new media such as YouTube create and affect user (inter)action with political discourse, texts and genres. Furthermore, drawing on CDA, it explores how
such interaction, in the form of text commenting, influences the medium itself by
(re)shaping social practices. Moreover, the study discusses and analyses the
important role played by the discourse community in this process. Finally, while
the genre of YouTube political speeches ostensibly represents a technologicallymediated public space (Wodak and Wright 2006) for political participation, the
openness of the medium and the social practices that this entails also allow for
the propagation of playful (and often discriminatory) language. In Section 2, the
main theoretical assumptions are introduced and briefly discussed. In Section 3
the corpora and methodology for the corpus-based analysis are introduced, while
in Section 4 the data are discussed and some preliminary conclusions provided.
2. I follow Fairclough (2003:3), who sees text as any actual instance of language usage, while
discourse is a more general way of representing the world (215).
248 Michael S. Boyd
2. Theoretical background
Critical discourse analysis and recontextualisation

For Critical Discourse Analysis (CDA), language is not only a product of social
interaction but also an important force in (re)shaping social practices, both positively and negatively (see, for example, Fairclough 1995, 2001, 2003, 2005; Van
Dijk 2001; Wodak 2008b; Wodak and Chilton 2007; Wodak and Meyer 2009).
Thus, discourse practices are conditioned by and they influence the type of social
activity, or genre, often leading to the creation of new or hybrid genres (Fairclough
1995, 2010). CDA is also interested in both production and reception conditions
and how these are reflected in social practices and genres (Fetzer and Johansson
2008). Moreover, with its focus on power, context and ideology (cf. Wodak 2008a:
297), CDA naturally lends itself to the investigation of the ways in which domination is embedded in and mediated through language use (Ietcu 2006:75). Finally,
in the literature ideology is often closely related to identity. Indeed, Fairclough
(2010:79) sees ideology as a relation between meaning (and therefore texts)
and social relations of power and domination. In other words, the way people
talk about themselves and others, both positively and negatively, reflect deeply
ingrained power relations, and the texts they produce can serve to sustain or
change ideologies (Fairclough 2003:9).
An important overarching concept in many CDA approaches is recontextualisation, which has been defined as one of the most effective tools for text production and interaction with other texts (Wodak and De Cillia 2007:323; see
also Van Leeuwen 2008)3. Put simply, recontextualisation is a process whereby
an element or argument (discourse) is extracted from one, often dominant, context or text for some strategic (or ideological) purpose (Chilton and Schffner
2002:17) and reproposed in a new one. Such relocation involves both the
suppression and filtering of meanings: recontextualisation entails suppression of some of the meaning potential of a discourse in the process of classifying discourses, establishing particular insulations between them (Chouliaraki
and Fairclough 1999:126). Chouliaraki and Fairclough (1999), thus, recognize
the twofold nature of recontextualisation, and accordingly position it within a
colonalisation-appropriation dialect. In such a view, recontextualisation has a
potentially colonising external presence which is however potentially appropriated and domesticated (Fairclough 2010:76). Indeed, recontextualisation can
3. The term comes from Bernstein and his sociological analysis of pedagogy. For more on its
adoption into CDA see Chouliaraki and Fairclough (1999) and Fairclough (1995, 2010).
be a powerful tool in transforming social or discursive practices and creating new

ones (Busch 2006:613), and may generate struggle[s] over forms of appropriation between social groups pursuing different strategies within the recontextualised context (Fairclough 2010:77). Seen this way, recontextualisation, is closely
tied to the underlying notions of power, context, ideology and identity. Crucial
for the present analysis, furthermore, is that recontextualisation can lead to what
Fairclough calls a re-imagining of a field or practice:
A discourse is decontextualised from its dialectical relationship with other elements of a field or network of social practices becomes an imaginary, often working in a metaphorical way in the re-imagining of aspects of the field or practices
it is recontextualised within (e.g., re-imagining student-academic relations in
higher education as consumer-producer relations), and, of course, open to enactment, inculcation and materialisation.
(2010:79)
One of the premises of this work is that text commenting leads to the re-imagining
of the genre of the political speech and, more generally, political communication.
In other words, there is a re-imagining of politician-audience relations in the context of the public sphere as a producer-user (community) relationship in a sphere
where the distinction between the public and the private sphere is blurred. Since
political differences have generally been construed and understood through linguistic means (Fairclough 2003:3), it is particularly fruitful to study how discursive
practices are recontextualised through various genres and political fields (Wodak
2008a:296) and ultimately adopted by new actors, discourses, and situations
(Wodak and Wright 2006:254). This chapter argues that the multimodal features
afforded by YouTube and the implications this has on user identity and participation offer new potentials for recontextualisation (Section 2.3). Specifically, through
the use of text comments, members of the YouTube discourse community selectively recontextualise elements of the individual speeches, Obamas discourse, and
other (often negative) discourses in their own textual realizations when posting
comments. The empirical data demonstrate that text comments (and the discourse
community that produces them) exhibit different types of recontextualisation:
on the one hand, the language of Obamas speeches (texts) and political message
(discourse) are recontextualised for positive presentation; on the other, playful and
discriminatory language are used to recontextualise the social practices of new
media which allow for the emergence of various discourses from the private into
the public sphere, be they of a playful and/or discriminatory nature.
250 Michael S. Boyd
Political speeches
Since speeches are often considered to be the most salient genre in political discourse analysis (Wodak 2009:2), it is not surprising that they have been the focus
of a number of studies (see, for example, Reisigl 2008; Sauer 1996; Schffner 1996;
Wodak and De Cillia 2007). Reisigl (2008:243) defines the speech as [a communicative act] uttered on a special occasion for a special purpose by a single person,
and addressed more or less to a specific audience. Some speeches have become
famous in a specific historical, political context (Chilton and Schffner 2002:21)
and have thus become historical events in themselves, which is certainly the case
of the three speeches analysed here. Generally, the political speech can be characterized as an Oratorical/Stable genre (Myers 2008:140) making it well suited for
fine-tuned analysis, but there are many different subgenres, which serve different
functions, as we see below.
The first speech, A More Perfect Union, is a typical (pre-)election speech,
one of the central subgenres (Myers 2008:143) in the field of political advertising, marketing and propaganda. Such speeches are generally persuasive and
serve political actors to assert [themselves] against political opponents as well
as to mobilise potential voters and party supporters for the speaker and his or
her party (Reisigl 2008:246,253). Such aspects are generally missing from the
subgenres of victory speech and inaugural address because at that stage in the election process, the speaker is no longer concerned with persuading voters, who have
already voted. The victory speech is best suited to establish inter-state relations
and relations with political opponents (Reisigl 2008:249), since such speeches
are generally targeted towards supporters. Moreover, such speeches share many
characteristics with the inaugural address. as they are used to signal the beginning
of the process of investiture (Campbell and Jamieson 2008:23). Finally, in the
US political system, the inaugural address is a highly ritualised event [] characterised by an elaborated and conventionally scripted orality rich in metaphors,
and is one of the most significant subgenres to be used in national politics (Reisigl
2008:252253). Campbell and Jamieson (2008:12) describe the president-elects
role in this genre as almost priestly in that the speaker must represent the nation
before God and demonstrate himself/herself to be the custodian of national and
supra-national values. Despite such high-minded intentions, however, the symbolic functions of inaugural addresses are often misunderstood and are easily criticised (Campbell and Jamieson 2008:2956).
Speeches, as all political discourse, should always be studied in their wider
context and on the basis of the other texts, genres and discourses they recontextualise (cf. Sauer 1996), It is therefore interesting to analyse some of the contextual
discourse ensuing from the speeches when rebroadcast on YouTube. In fact, the
public response that followed the rebroadcasting of Obamas speeches on YouTube

can be seen as leading to the creation of a separate (emerging) genre (Boyd 2009b,
2011). Even though the original speech events remain unchanged, by rebroadcasting them on YouTube the reception criteria are indelibly modified through the
multimodal affordances and interaction with the YouTube community. When a
political speech is rebroadcast the discursive and social practices are recontextualised. Given the importance of the context of speeches to their function and impact,
the new discourse can be characterized as a new (hybrid) genre.
New media, YouTube and political speeches

The second generation of web technologies, commonly known as Web 2.0, has
dramatically altered the production and reception of web-based genres, texts and
discourses. Such technologies include, among others, blogging, social networking, Wikis, Moodles, social bookmarking, podcasting and, of course, YouTube.
Web 2.0 applications allow users to become co-creative participants (Lister et
al. 2009:204) who consume and (re)produce new texts, discourses and genres.
Multimodality, long a feature of new media, has been enhanced by a combination
of modes, such as, e.g., writing and images, media of communication, and new
affordances, which provide users with potentials for representational and communicational action (Kress 2002:5). All of this has had a considerable impact on
users, who are involved in creating forms of action and interaction in the social
world, new kinds of social relationships and new ways of relating to others and
oneself (Thompson 1995:4). In other words, new media are characterized by a
high degree of intertextuality as they draw on other texts and play with established conventions of form and representation (McKay 2006:600), and interactivity as they create new types of social interaction, further differentiating them from
the texts and genres from which they originate.
The ability to post comments has been described as a way to expand the
potential for text production (Savoie 2009:182). This function, which is common
to many new media, such as blogs (Gryc and Moilanen, this volume), is a truly
interactive feature, whose full potential has yet to be fully exploited (Herring et
al. 2004:11). Hess (2009:427) observes that YouTube comments, which are full
of examples of web-speak, should be interpreted as more playful than serious. However, these comments can also have a negative side, namely when they
become purveyors of racist, sexist, and extreme nationalist ideas and other forms
of hate speech (Thornborrow 2006:618).
An important aspect of these online communities is identity, which is
expressed or repressed among users in various ways. First, by being able to create
252 Michael S. Boyd
virtual IDs or pseudonyms, on-line users can conceal relevant aspects of their identity such as age, sex, gender, class, and ethnic background and construct a person
which is free from visual or spoken identity markers (Thornborrow 2006:618).
In addition to a sense of equality among speakers, Bentivegna (2002:53) proposes
two other distinctive features characterising discourse behaviour in new media:
anonymous reference to personal experience and to the information offered by the
entire media system are employed to construct frames of reference within which
topics are introduced. The creation and use of pseudonyms, however, can also have
negative effects. Hess, for example, sees them as an obstacle to true communication, as they hinder the ability to mobilize due to their flippant and anonymous
comportment (2009:427). Furthermore, virtual identities can be used to purposely
deceive others (Thornborrow 2006:618). Finally, as the data from this study indicate, pseudonyms themselves can be used to propagate racism (Section 5).
YouTube is one of the most popular and influential social networks today.
Although primarily devised as a way to watch and share video over the Internet,
YouTube also offers a number of typical Web 2.0 affordances that registered
logged-in users can take advantage of when visiting the site (registrational interactivity, Listener et al. 2009). These include the possibility to flag videos, rate them
with a five-star system, post text and video comments, rate other users comments,
and embed videos on other sites. Hess sees the role of a community as crucial for
interaction: YouTubers frequently respond to each others videos, enacting spoofs
of originals or giving commentary to issues brought up by their peers (2009:414).
Since users must register as members with a user account (and screen name) and
be logged in to access the interactive features, a sense of community is established.
The 2008 US elections marked a turning point for YouTube in the political
sphere. Not only did the candidates use it for political advertising, but it became a
first-stop source for political everything (Heffernan 2008). The beginning of this
process of political YouTubification (May 2008) is closely tied to the September
2007 joint CNN-YouTube broadcast of the primary debates (Macnamara 2008:2).
It should be stressed that primary candidates had to embrace YouTubes purported
openness to be accepted by the YouTube community, which often meant relinquishing control over their videos by allowing the public to react, embed, critique,
recut them and also become the object of satire (Heffernan 2008). The Obama
campaign team appeared to acknowledge this notion, applying it not only to their
own videos but also allowing the proliferation of spoofs and mash-up videos,
which recontextualised discourses (and texts) by and about Obama, as exemplified, e.g., by the enormously popular videos Crush on Obama and Yes We Can.
At this point, we should ask if the interactive features on YouTube really do
create true interaction and participation. While it is true that YouTube users have
the possibility to participate in text creation (and recontextualisation) through
various interactive instruments, participation also implies that the sender of the
text envisages the interaction. This, however, is not necessarily the case for the
producers of the original video texts who, at least in the case of the three videos
studied here, are not directly involved in the participatory process (Carpentier
2009:411). Such interaction, then, I would argue, should be viewed as what
Thompson (1995:98) labels quasi-participation, and within which [n]either
producers nor recipients are under any mutual obligation to take account of the
responses of the other.
3. Data and methodology
Of the three Obama speeches analysed here, two different versions of the preelection speech and the Inaugural address posted on YouTube generated a significantly high number of views and comments. Therefore, a total of five videos were
used in the study and are shown in Table 1.4
Table 1. The YouTube Obama speeches (November 2010).
YouTube speech
Source
No. views No. text

No.
Average
comments ratings rating
A More Perfect Union 1

A More Perfect Union 2
Obama Election Victory Speech
Obama Inaugural Address 1
Obama Inaugural Address 2
barackobama.com
barackobama.com
barackobama.com
CSPAN
Whitehouse
6,330,137 10,032
1,699,387 1,339
1,362,691
225
4,298,366 59,764
1,352,964 6,894
29,015
12,927
4,771
18,095
3,471
4.5 stars
4.5 stars
5 stars
4.5 stars
4.5 stars
The versions of the speeches were chosen on the basis of the high number of views
and the reliability of the source. All of the comments displayed on the YouTube
pages (which display a maximum 500 comments per page) were copied and
pasted into text documents for further processing. Due to the large number of
comments separate text files were created to keep the documents from becoming
too unwieldy.5 Unnecessary symbols, spaces and formatting were then removed
4. Of the two versions, one is these is the cleaner and more elegantly produced tailor-made
YouTube version (Heffernan 2009). Although there is a similar version of Obamas victory
speech, with more than 5 million views, there are no text comments, and therefore was omitted
from the study.
5. A total of 10 texts were created (2 from the A More Perfect Union videos, 1 from Election
Victory Speech, and 7 from the two Inaugural Address videos). The files were later combined
when imported into WordSmith Tools 4.0.
254 Michael S. Boyd
from the documents. Next, the various text files were imported into WordSmith
Tools, version 4.0 (Scott 2006), creating the data for the three corpora (Table 2): A
More Perfect Union Comments (AMPUCom); Obama Election Victory Speech
Comments (OEVSCom); and Obama Inaugural Address Comments (OIACom).
Table 2. Comment corpora and reference corpus used in the study.
Corpus
Total comments
Tokens
Types
AMPUCom
OEVSCom
OIACom
Combined
Open ANC Reference
11,371
,225
66,658
78,254
8,832 (texts)
436,546
5,110
1,790,398
2,232,054
16,426,025
18,783
823
47,323
66,929
168,258
Word (frequency) lists, the number of tokens (running words) and the types
(distinct words) were then generated. The wordlists were subsequently compared
against a previously compiled reference wordlist (Open ANC Corpus6) for the
creation of three new keyword lists that form the basis of the discussion. Keyness7
is used to establish the salience (Baker 2006:125) or aboutness (Scott 2006) of
specific texts.
In the next step, the keyword lists were analysed qualitatively: they were
manually edited to include only the most salient items for this study. First, most
lexical items that were absent from the reference corpus were eliminated, such
as the commonly repeated words day, year and ago, which appear in the header
of the comment following the hyperlinked user name to indicate when the comment was posted, e.g., democratic hypocrite (9 months ago). Other grammatical
forms, such as prepositions, articles, determiners non-action verbs, auxiliaries,
etc., deemed insignificant for the study were eliminated. Lexical forms with a high
referential value, pronouns, most examples of web-speak, (e.g., dont and ur), username pseudonyms, and other terms related to Obama (e.g., Obama and Biden)
lacking in the reference corpus but considered relevant to the study were retained.
In this phase of the editing process, I was particularly interested in determining
6. The Open ANC corpus (available http://www.americannationalcorpus.org/OANC/) consists

of both spoken (3.2 million words) and written texts from various genres including face-to-face
conversation, telephone calls, government reports, travel guides, fiction, letters, and political
magazines.
7. Keyness is determined by comparing a word frequency list to another list to determine
which words occur statistically more often. A keyword list gives a measure of saliency, whereas
a simple word list only provides frequency (Baker 2006:125).
linguistic realizations that might be indicative of positive presentation and recontextualisation; pronouns, which are closely tied to the notions of power, identity
and ideology; and negative and discriminatory lexical items. The discussion of the
keywords was generally limited to the top 30 lexical items, with the exception of
the last corpus, for reasons we shall discuss below.
The final round of the analysis consisted of comparing the comment corpora
with the transcripts of the original speeches8 in order to determine recontextualised lexical items. In Section 4 only the first speech is reproduced and discussed.
The data for the speech corpora are listed in Table 3.
Table 3. Individual speech corpora.
Speech
Tokens
Types
A More Perfect Union

Victory Speech
Inauguration Speech
4,883
4,853
2,402
1,386
1,194
,894
4. Data analysis
Before moving on to the discussion of the various corpora, Table 4 provides an
overview of the average number of words per comment in the corpora.9
Table 4. Words per comment.
Corpus
Average number of tokens/comment
AMPUCom
OEVSCom
OIACom
Combined
38.4
22.7
26.9
29.3
These data indicate some preliminary differences. First, the average number of
words per comment in the AMPUCom is considerably higher than in the other
two corpora. The different speeches seem to generate different amounts of commenting. Second, the lower comment rate in OEVSCom (22.7) can be interpreted
8. All of the transcripts were taken from the New York Times website (www.nytimes.com).
9. It should be noted that the number of posts by the same user was not taken into consideration. Furthermore, the tokens provided in Table 2 also include user names and other information from the headers. Users who posted a high number of comments, however, may be evinced
from the Keyword lists, as discussed below.
256 Michael S. Boyd
in light of the low number of types in Table 2 (823). A possible explanation for
the fewer comments emerges in the list of raw word frequencies in OEVSCom
(Table 5).
Table 5. The 20 most frequent words in OEVSCom.
N
Word
Freq
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
#
AGO
OBAMA
YEAR
THE
OF
AND
WON
IF
COPY
PASTE
WE
I
A
GLAD
TO
YOU
YOURE
PRESIDENT
HAS
638
225
218
214
152
88
84
75
66
61
61
61
60
58
46
46
45
37
36
35
12.5
4.4
4.3
4.2
3.0
1.7
1.6
1.5
1.3
1.2
1.2
1.2
1.2
1.1
0.9
0.9
0.9
0.7
0.7
0.7
The most frequent word by far is #, a symbol generally used in WordSmith Tools
to indicate lemmas containing numbers or symbols (Scott 2006).10 The unusually
high percentage (12.5%) is almost three times as great as the next entry. While
it is true that many numbers do appear in the corpus, this does not explain the
12.5%. Furthermore, the next entries on the wordlist are relatively predictable:
ago can be explained by its use in all posts to refer to the time frame along with
year in the common 1 year ago; Obama is to be expected as are the common
grammatical words the, of, and. Some of the lexical items, however, beckon further investigation, e.g., won, if, copy, paste and glad. A concordance was generated
to determine where the numbers are located in the text, an example of which is
provided in Figure 1.
10. The programme was used on a MacBook running Windows XP through the virtualization
software VMWare Fusion (v. 3.0).
Figure 1. Concordance lines from OEVSCom with #.
Although the original comment is not entirely clear here, many of the items in
Table 5 are part of the concordance. In the full concordance there are 601 instances
of such combined text/number/symbol usage, thereby justifying the high percentage of # in the frequency list. To determine what this text, numbers, and symbols
referred to I had to return to the original YouTube comments page. Figure 2 shows
the original.
Figure 2. Example of copy and paste text/number/symbol Obama victory message.
The original text file contains 64 uses of Figure 1 or a variation thereof. A further qualitative look revealed mainly positive comments. Thus, while the victory
speech generated comparatively few text comments, the comments were used to
positively recontextualise the speech. This hypothesis is further tested in the discussion of the keywords below.
Table 6 presents the key words, ordered according to keyness, in OEVSCom
as compared to the OpenANC reference corpus.
Not surprisingly, many of the top items are from the text/number/symbol victory message discussed above. Thus, paste, #, won, copy, glad, ur (web-speak for
you are), dream, begun are all part of 2008 / Copy and paste if ur glad Obama
won / the start of the dream has begun!!! The keywords also contain two evident examples of user names: hyernespreng and phylosophycal wisdom. Most of
the other items, however, including the two pseudonyms have a relative frequency
which is too low to be considered important. The two other lexical items of interest
for this study are the personal pronoun we and change, both statistically relevant
in Obamas discourse (Boyd 2009b) and examples of (positive) recontextualisation.
The lexical item months, part of the x months ago formula discussed above, is
of limited interest. Thus, the keyword analysis offers further evidence of positive
forms of recontextualisation.
258 Michael S. Boyd
Table 6. Keywords OEVSCom vs. OpenANC.

N
Key word
Freq
RC Freq
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
OBAMA
PASTE
#
WON
COPY
GLAD
BARACK
UR
DREAM
BLESS
BEGUN
HAPPY
GOD
YOURE
PRESIDENT
IF
YES
AMERICA
CONGRATULATIONS
START
MR
BIDEN
WE
HJERNESPRENG
PHYLOSOPHYCALWISDOM
CONGRATS
THANK
MONTHS
CHANGE
HES
218
61
638
75
61
46
21
25
33
18
23
26
27
37
36
66
33
25
11
23
14
6
61
4
3
3
9
13
15
3
4.27
1.19
12.50
1.47
1.19
0.90
0.41
0.49
0.65
0.35
0.45
0.51
0.53
0.72
0.70
1.29
0.65
0.49
0.22
0.45
0.27
0.12
1.19
0.08
0.06
0.06
0.18
0.25
0.29
0.06
0
39
616489
929
824
364
0
46
406
72
429
1030
1524
5658
6231
33072
5466
2214
49
2822
1379
22
55704
0
0
0
745
2724
4440
4
RC % Keyness
3.75
0.03
0.04
0.20
0.03
0.01
0.02
0.34
0.02
0.03
3530.452393
852.244751
682.7085571
679.644104
542.4664307
455.721283
339.2672119
311.8158875
299.2006531
200.7620697
190.0471649
176.7379761
164.9041748
154.9130554
142.277359
134.4335938
133.1455841
130.8145752
120.5509872
105.9017181
70.35123444
67.83305359
66.53504944
64.60900116
48.45616531
48.45616531
48.2425499
46.66303253
44.30900574
38.89793777
The next set of keywords to be discussed is taken from AMPUCom. Table 7

demonstrates that many of the keywords can be interpreted as examples of (positive) recontextualisation. First, two of the lexical items, hope (N8) and change
(N15), are recontextualised from Obamas campaign discourse and message (Boyd
2010), while other items such as pastor (N30), race (N11), and wright (N28), are
recontextualisations from the speech and/or media discourses about the speech.
Second, if we recall the main reason for A More Perfect Union, i.e., for Obama to
address the growing criticism over his relations with his former pastor, Jeremiah
Wright and his controversial statements (Boyd 2009a) some of the other lexical
Table 7. Keywords AMPUCom vs. OpenANC.

N
Key word
Freq
RC Freq
RC % Keyness
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
HE
THIS
MAN
AMERICA
VOTE
SPEECH
PRESIDENT
HOPE
COUNTRY
HIM
RACE
SPEECHES
GREAT
AMAZING
CHANGE
IS
POLITICIAN
I
WILL
US
BLESS
GOD
AM
OUR
THANK
HIS
WRIGHT
HILLARY
PASTOR
AMERICAN
5470
6377
2000
1640
1352
3289
1906
1123
1193
1616
683
401
1106
489
933
7099
385
7384
2008
1155
306
583
839
1664
422
2248
293
389
225
950
1.25
1.46
0.46
0.38
0.31
0.75
0.44
0.26
0.27
0.37
0.16
0.09
0.25
0.11
0.21
1.63
0.09
1.69
0.46
0.26
0.07
0.13
0.19
0.38
0.10
0.51
0.07
0.09
0.05
0.22
39021
61479
4019
2214
1497
22298
6231
1673
4047
9781
1383
158
5952
543
4440
147550
225
155783
22346
8231
72
1524
4224
17968
745
31241
157
634
21
7017
0.24
0.37
0.02
0.01
0.14
0.04
0.01
0.02
0.06
0.04
0.03
0.90
0.94
0.13
0.05
0.026
0.11
0.19
0.04
8896.984375
7584.032715
7181.12793
6849.742188
6020.510254
5591.123535
5403.819336
4529.97168
3311.304443
3022.231689
2442.638916
2273.41626
2268.198975
2174.72998
2092.111816
2028.043335
2022.494141
2015.207275
1983.167358
1873.012817
1872.099731
1855.675415
1807.315796
1710.011597
1596.155273
1585.470215
1567.561157
1517.417969
1501.987183
1489.455811
items can be explained as recontextualisations. Third, the pronoun he (N1) is the

highest ranking keyword followed by the demonstrative this (N2) and the noun
man (N3). The collocates and clusters11 in the keyword list show that these are
11. For those unfamiliar with the software, it is possible to create concordances of selected
words in the keyword lists. Once a concordance has been generated it is then possible to see the
collocates on either side of the selected item (L1, R1, e.g.) as well as the most common clusters
(repeated patterns of words found next to the selected word). In this case the clusters were
limited to three words.
260 Michael S. Boyd
mostly used for reference to Obama. In fact, the most common L1 collocations12
of man, are this (884) and a (259), which is further clarified by looking at the most
frequent three-word clusters, this man is (210), a man who (61), a great man (53),
elect this man (43), vote for this [man] (41), etc. Finally, the two adjectives great
(N13) and amazing (N14) clearly represent positive presentation, which is supported by the collocate and cluster data: great speech (268), great speeches (27) and
the misspelled great speach (15), which also explain the high incidence of speech
(N6) and speeches (N12) in the keywords; great man (84), great president (51), great
speaker (47), great leader (47), great country (23); amazing speech (96), amazing
man (30); and the adverb + adjective forms simply amazing (28), truly amazing
(14), absolutely amazing (13), just amazing (12). Finally, the frequent use of the
1st-person singular pronoun (N18) is significant, especially when compared with
Obamas tendency to use the 1st-person plural pronoun. I would argue that its
high frequency in the comments demonstrates personal participation and commitment on behalf of the commentators. In general, however, these data point to
an overall positive character. In fact, in the first thirty keywords there are no examples of web speak, playful and/or discriminatory discourse practices, which is
even more remarkable when compared to the OIACom as we shall see below. But
before looking at the final corpus, it would be insightful to compare AMPUCom
with the original speech to find further support for positive recontextualisation.
The data in Table 8 would appear to corroborate the generally positive nature
of the comments elicited in Table 7. Although a full discussion of the data is
impossible due to space limitations, some of the lexical items deserve comment.
Briefly, on the basis of a previous study of Obamas campaign speeches and his
turns in the presidential debates (Boyd 2009b), I would argue that the most salient
forms for recontextualisation are the 1st- person Pl pronominal forms we (N1),
our (N12), us (N16), the (ostensibly) modal forms will (N4), can (N6), if (N10),
need (27) and the nouns hope (N13) and change (N18). Even though a full analysis
of the concordance data would be necessary to determine actual usage (whether
positive or negative), these 9 forms, which have all been demonstrated to be possible indicators of recontextualisation (Boyd 2009b), account for 3.76% of overall
usage in the comments. Moreover, a cursory glance at the other keywords again
shows no examples of negatively-loaded or playful language. Thus, the data in
Table 7 appear to offer further support for the generally positive nature of the
comments in AMPUCom. We could tentatively assert, then, that this type of (historically important) campaign speech generates, for the most part, positive recontextualisation strategies.
12. L1 refers to the first word to the left of the queried item.
Table 8. Keywords AMPUCom vs. A More Perfect Union (speech).

N
Key word
Freq
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
WE
HIS
MAN
WILL
HAVE
CAN
MY
AMERICA
PEOPLE
IF
HIM
OUR
HOPE
GREAT
COUNTRY
US
WORLD
CHANGE
YOUR
ME
TIME
AMERICAN
SEE
WORDS
WHITE
KNOW
NEED
IM
THEY
GOOD
405
342
269
268
288
270
254
227
197
190
180
209
175
144
151
144
130
133
130
117
121
106
86
87
105
84
87
81
96
78
0.81
0.68
0.54
0.53
0.57
0.54
0.51
0.45
0.39
0.38
0.36
0.42
0.35
0.29
0.30
0.29
0.26
0.27
0.26
0.23
0.24
0.21
0.17
0.17
0.21
0.17
0.17
0.16
0.19
0.16
RC Freq
66
9
9
9
36
23
31
10
14
10
8
41
7
3
15
14
3
6
4
12
17
20
3
4
27
4
7
3
25
4
RC % Keyness
1.35
0.18
0.18
0.18
0.74
0.47
0.63
0.20
0.29
0.20
0.16
0.84
0.14
0.06
0.31
0.29
0.06
0.12
0.08
0.25
0.35
0.41
0.06
0.08
0.55
0.08
0.14
0.06
0.51
0.08
1623.034302
1596.931641
1238.974487
1234.093262
1191.190796
1164.638916
1052.103149
1028.285889
861.1304932
849.5949097
813.333313
804.8853149
795.4228516
672.713623
637.7473145
609.409729
604.777832
599.2348022
597.6694946
491.6701355
488.9660339
408.9632874
392.2913818
390.7704773
381.8666992
376.4163208
374.3079834
368.2672424
348.042572
347.7482605
Finally, we look at the keywords from the OIACom, which paint a very different picture of the text commenting and recontextualisation practices. The data are
provided in Table 9. The first thing to emerge from these data is the high incidence
of obscenities and one example of racist epithet: fuck (N9), shit (14), fucking (N17),
ass (N25) and nigger (N27). What is even more surprising is their high frequency
in the OIACom corpus compared to their absence in the other two corpora.
While the obscenities appear to be examples of what Hess calls YouTubes inherent
informal and playful nature, the last example is clearly used for discriminatory
262 Michael S. Boyd
Table 9. Keywords OIACom vs. OpenANC.

N
Key word
Freq
RC Freq
RC % Keyness
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
HE
PRESIDENT
YOU
BLACK
BUSH
LOL
RACIST
AMERICA
FUCK
YOUR
PEOPLE
DONT
GOD
SHIT
WHITE
COUNTRY
FUCKING
HIM
MAN
WILL
COMMENT
ASS
IM
HOPE
HES
HATE
NIGGER
STUPID
BLESS
WORLD
16624
7461
27145
4798
4318
2517
2651
3947
2456
8042
8061
1798
2810
1809
3967
3693
1581
5200
3414
7316
2120
1363
1229
2193
1044
1563
1006
1295
1059
3603
0.93
0.42
1.52
0.27
0.24
0.15
0.15
0.22
0.13
0.45
0.45
0.10
0.16
0.10
0.22
0.22
0.09
0.30
0.19
0.41
0.12
0.08
0.07
0.12
0.06
0.09
0.06
0.07
0.06
0.20
39021
6231
108165
3038
2175
2
223
2214
128
16688
18603
0
1524
118
4789
4047
39
9781
4019
22346
1387
163
51
1673
4
715
17
323
72
7566
0.24
0.04
0.66
0.02
0.01
0.01
0.10
0.11
0.03
0.02
0.06
0.02
0.14
0.01
0.05
17418.07031
17059.31641
12798.55859
12435.97852
12212.48242
11649.35742
10781.31641
10731.35352
10405.91992
9590.079102
8588.561523
8343.9375
7735.808594
7531.608887
7341.750977
7263.661621
6977.070801
6814.673828
6421.19873
5440.710938
5417.862793
5321.563477
5284.92041
5233.962891
4792.766113
4566.438965
4498.611816
4458.499023
4393.052734
4240.228027
purposes and extreme racism. In fact, if we look further down the list of keywords
it is easy to find other examples of evident racism (Table 10).
The examples in Table 10 provide further evidence for a high incidence
obscenities and loaded (racial) language in the comments. While some of these
terms mean very little in isolation, a closer look reveals racist discourse practice.
To understand the first entry in 10, Hussein, we should recall that Obamas middle
name is Hussein, which according to many of his detractors is the proof that he
is really a Muslim (offering a possible explanation to the appearance of Muslim
[N47]). The three-word clusters associated with Hussein, generally corroborate its
Table 10. Possible examples of discriminatory language in OIACom.

N
Key word
Freq
35
40
43
46
47
51
53
56
63
66
71
72
74
80
83
HUSSEIN
IGNORANT
INSANE
SOCIALIST
MUSLIM
MORON
IDIOT
HELL
BITCH
LIBERAL
FUCKED
MESSED
DUMB
DAMN
RACISM
1215
929
876
853
1016
690
763
923
627
971
457
513
602
575
578
0.07
0.05
0.05
0.05
0.06
0.04
0.04
0.05
0.04
0.05
0.03
0.03
0.03
0.03
0.03
RC Freq
314
99
99
108
335
19
82
430
87
850
9
55
204
179
200
RC % Keyness
4150.302246
3679.742676
3444.823242
3304.905273
3270.449463
3030.587891
3019.077637
2680.051514
2398.124268
2165.08667
2033.472046
2030.39209
1923.585815
1878.579346
1836.454102
negative usage: insane Hussein is (130), Barack Hussein Obama (124), Hussein is
a (82), deport insane Hussein (80), that insane Hussein (65), etc. Such examples
demonstrate that the term is mainly used for negative and racist descriptions and
are therefore negative recontextualisations.
The keywords in Table 9, however, are not all negative. First, there are four
more examples of web-speak (and, therefore, representative of playful behaviour):
LOL (N6), dont (N12), im (N23) and hes (N25). Second, there are a number of
words that can be used for either positive or negative presentation: black (N4),
racist (N7) and white (N15). A cursory glance at the concordance data supports
such an observation, although further research is necessary to determine the
percentages. Also significant are the clearly religious words god (N13) and bless
(N28), which often collocate together in god bless America (311) and can therefore also be seen as positive presentation. Finally, the presence of hope (N24) is
an example of positive recontextualisation from Obamas discourse, as has been
demonstrated in the discussion above of A More Perfect Union. Thus, the data
appear to demonstrate different types of behaviour among the commenters. On
the one hand, there is a high incidence of playful, obscene and even racist language that is clearly used for negative recontextualisation, while, on the other,
some of the language used can be exploited for both negative and positive presentation or for evidently positive recontextualisation. Of the three speeches, the
Inaugural Address generates not only the greatest number of comments but also
the most varied types of discourse practice.
264 Michael S. Boyd
5. Discussion and conclusion

The discussion of the corpora data has highlighted some important differences in
the text commenting. On the one hand, the comments of A More Perfect Union
and Obama Election Victory Speech demonstrate a relatively high incidence
of positive presentation and recontextualisation, but of different types. While A
More Perfect Union appears to have generated a much greater number and more
varied types of positive recontextualisation, the Obama Election Victory Speech
comments are dominated by a single message that was shared among users. This
message combines the playful and interactive features of web participation and
co-creativity with positive recontextualisation strategies, and it is a clear example of new kinds of discourse practice fostered by the web and the new forms
of interaction it can create. The fact that the Election Victory Speech generated
comparatively few and similar comments can possibly be attributed to the fleeting
nature of this subgenre of speech, which is addressed mainly to supporters and,
therefore, to those who want to celebrate the moment of victory. Moreover, in US
politics the victory speech is soon eclipsed by the more official Inaugural address.
In the case of A More Perfect Union a comparison of the word frequencies in the
comments with those in the original speech provides further evidence for positive
recontextualisation strategies: elements from the original speech and Obamas discourse have been appropriated and domesticated by users in their text comments.
While such behaviour is most likely due to the social and historical importance
of the speech, the role of Obamas communicative skills, the support of (especially
young) voters, the positive media reception and the important themes addressed
by the speech should not be underestimated. The comments of the Inaugural
Speech, on the other hand, while containing some forms that could be used for
positive presentation, are also full of examples of web-speak, obscene language
and even racist epithets. Such usage, I would argue, is most likely the result of
the symbolic and ritualised nature of the subgenre, which is often misunderstood
becoming the object of criticism and derision. In addition, this subgenre of speech
is closely tied to the process of investiture and therefore to the President himself,
so it is natural that it will attract both supporters and detractors.
Since the three subgenres of speech studied here generate different types of
comments, we can assume that there are some important differences between
the user communities and the reception factors of the original texts (speeches).
Even though YouTubers relocate discourses, both positively and negatively, when
posting comments to political speeches, as demonstrated by the three speeches
analysed here, they selectively filter elements for their own strategic purposes
which are influenced by factors of identity and ideology. While positive recontextualisation appears to be a common strategy in all three subgenres of speech, it is
more widespread in the subgenres of election and victory speeches, and, therefore,
within area of political advertising, marketing and propaganda. Through commenting and the types of recontextualisation that users adopt, we can discern
some features of what Fairclough (2010) calls re-imagining in the field of politics: the typical politician-audience relationship is re-imagined as a relationship
between the (video) producer and the (YouTube) user, and the distinction between
the public and the private is often blurred. Furthermore, the playful perception
that users have of YouTube and the widespread anonymity among users allow
them to say whatever they want often with little self-control and respect for typical politeness strategies, thereby appropriating elements of the private sphere (i.e.,
recontexualising) into their online discourse practices. Thus, recontextualisation
processes also help to influence and transform discursive and social practice of the
medium. If YouTube is really to be a technologically-mediated public space for
political participation, however, text producers (i.e., both political actors and commentators) need to understand that openness may also allow for the propagation
of discriminatory and racist discourses. Indeed, future research should be aimed
at understanding how such processes work and how users interact with each other
in their text comments and through the use of other affordances of the medium.
References
Baker, P. 2006. Using Corpora in Discourse Analysis. London: Continuum.
Bentivegna, S. 2002. Politics and new media. In Lievrouw, L.A. and S. Livingstone (Eds), Handbook of New Media. London: Sage, pp. 5061.
Boyd, M.S. 2009a. De-constructing race and identity in US presidential discourse: Barack
Obamas speech on race. Atlantis. Journal of the Spanish Association of Anglo-American
Studies 31, pp. 7594.
Boyd, M.S. 2009b. Regrammaticalization as a restrategization device in political discourse.
Paper presented at Corpus Linguistics 2009. University of Liverpool, 2123 July 2009.
Boyd, M.S. 2010. Changing frames in political discourse: hope and change in the US presidential primaries. In A. Duszak, J. House and . Kumiga (Eds.), Globalization, Discourse,
Media: In a Critical Perspective (Globalisierung, Diskurse, Medien: Eine Kritische Perspektive). Warsaw: Warsaw University Press, pp. 343367.
Boyd, M.S. 2011. (New) political genres for the masses? YouTube in the 2008 US presidential
elections. In S. Sarangi, V. Polese, and G. Caliendo (eds.), Genres on the Move: Hybridization and Discourse Change in Specialized Communication. Naples: Edizioni Scientifiche
Italiane, pp. 2744.
Busch, B. 2006. Media, politics, and discourse: Interactions. In K. Brown (ed.), Encyclopedia of
Language and Linguistics. Oxford: Elsevier, pp. 609616.
Campbell, K.K. and K.H. Jamieson. 2008. Presidents Creating the Presidency. Deeds Done in
Words. Chicago/London: University of Chicago Press.
266 Michael S. Boyd
Carpentier, N. 2009. Participation is not enough: The conditions of possibility of mediated

participatory practices. European Journal of Communication 24, pp. 407420. DOI:
10.1177/0267323109345682
Chilton, P. and C. Schffner. 2002. Introduction: Themes and principles in the analysis of political discourse. In P. Chilton and C. Schffner (eds.), Politics as Text and Talk. Analytic
Approaches to Politcal Discourse. Amsterdam: John Benjamins, pp. 141. DOI: 10.1075/
dapsac.4
Chouliaraki, L. and N. Fairclough. 1999. Discourse in Late Moderinity. Rethinking Critical Discourse Analysis. Edinburgh: Edinburgh Univeristy Press.
Fairclough, N. 1995. Critical Discourse Analysis: The Critical Study of Language, London/New
York: Longman.
Fairclough, N. 2001. Language and Power, Harlow/New York: Longman.
Fairclough, N. 2003. Analysing Discourse: Textual Analysis for Social Research. London/New
York: Routledge.
Fairclough, N. 2005. Critical Discourse Analysis in transdisciplinary research. In Wodak, R.
and P. Chilton (eds.), A New Agenda in (Critical) Discourse Analysis. Amsterdam: John
Benjamins, pp. 5370.
Fairclough, N. 2010. Critical Discourse Analysis: The Critical Study of Language. Harlow:
Longman.
Fetzer, A. and M. Johansson. 2008. Hybridities in political media discourse. Politics and Culture, 4. Available: http://aspen.conncoll.edu/politicsandculture/adpage.html (Accessed 15
February 2010).
Grove, S. 2008. YouTube: The flattening of politics. Nieman Reports. Available: http://www.nieman.harvard.edu/reportsitem.aspx?id=100019 (Accessed 15 November 2009).
Heffernan, V. 2008. Clicking and choosing. The New York Times. New York. Available: http://
www.nytimes.com/2008/11/16/magazine/16wwln-medium-t.html (Accessed 30 November
2009).
Heffernan, V. 2009. The YouTube presidency - Why the Obama administration uploads so much
video. The New York Times. New York, Available: http://www.nytimes.com/2009/04/12/
magazine/12wwln-medium-t.html? (Accessed 11 April 2009).
Herring, S.C., L.A. Scheidt, S. Bonus and E. Wright. 2004. Bridgining the gap: A genre analysis
of weblogs. Proceedings of the 37th Hawaii International Conference on System Sciences,
2004, pp. 111.
Hess, A. 2009. Resistance up in smoke: Analyzing the limitations of deliberation on YouTube. Critical Studies in Media Communication 26, pp. 411434. DOI: 10.1080/15295030903325347
Ietcu, I. 2006. Discourse Analysis and Argumentation Theory. Bucharest: Editura Universitatii
din Bucaresti.
Kress, G. 2002. Literacy in the New Media Age. London: Routledge.
Lister, M., J. Dovey, S. Giddings, I. Grant and K. Kelly. 2009. New Media: A Critical Introduction.
Abingdon, Oxon: Routledge.
Macnamara, J. 2008. E-electioneering: Use of new media in the 2007 australian federal election.
In Elspeth Tilley (Ed.), Power & Place: Refereed Proceedings of the Australian & New Zealand Communication Association Conference, Wellington, July 9-11. Available: http://www.
massey.ac.nz/?aaac11238s (Accessed 15 November 2009).
May, A.L. 2008. Campaign 2008: Its on YouTube. Nieman Reports. Available: http://www.nieman.harvard.edu/reportsitem.aspx?id=100018 (Accessed 15 November 2009).
McKay, S. 2006. Media and language: Overview. In K. Brown (ed.), Encyclopedia of Language
and Linguistics, Vol. 7. Oxford: Elsevier, pp. 597602.
Myers, G. 2008. Analyzing interaction in broadcast debates. In R. Wodak and M. Krzyanowski
(eds.), Qualitative Discourse Analysis in the Social Sciences. Basingstoke/New York: Palgrave
Macmillan, pp. 121144.
Nagourney, A. 2008. The 08 campaign: Sea change for politics as we know it. The New York
Times. New York, Available: http://www.nytimes.com/2008/11/04/us/politics/04memo.
html?_r=1&scp=1&sq=a%20sea%20change%20for%20politics%20as%20we%20know%20
it&st=cse (Accessed 15 November 2009).
Reisigl, M. 2008. Rhetoric of political speeches. In R. Wodak and V. Koller (eds.), Handbook of
Communication in the Public Sphere. Berlin/New York: Mouton de Gruyter, pp. 243269.
Reisigl, M. and R. Wodak. 2009. The discourse-historical approach (DHA). In R. Wodak and
M. Meyer (eds.), Methods of Critical Discourse Analysis. Second ed. London: Sage, pp. 87121.
Sauer, C. 1996. Echoes from abroad speeches for the domestic audience: Queen beatrix
address to the Israeli parliament. Current Issues in Language & Society, 3, pp. 233267.
Savoie, H. 2009. John McCain get Barackrolld: Authorship, culture and community on YouTube. YouTube and the 2008 Election Cycle in the United States. University of Massachusetts
Amherst: ScholarWorks@UMass, pp. 177188. Available: http://scholarworks.umass.edu/
jitpc2009/1 (Accessed 15 November 2009).
Schffner, C. 1996. Editorial: Political speeches and discourse analysis. Current Issues in Language & Society, 3, pp. 201204.
Scott, M. 2006. Oxford Wordsmith Tools Version 4.0. Oxford: Oxford University Press. Available:
http://www.lexically.net/wordsmith/
Thompson, J.B. 1995. The Media and Modernity: A Social Theory of the Media. Cambridge:
Polity Press.
Thornborrow, J. 2006. Media: Analysis and methods. In K. Brown (ed.), Encyclopedia of Language and Linguistics, Vol 7. Oxford: Elsevier, pp. 616623.
Van Dijk, T.A. 2001. Multidisciplinary CDA: A plea for diversity. In R. Wodak and M. Meyer
(eds.), Methods of Critical Discourse Analysis.London: Sage, pp. 95120. DOI: 10.4135/
9780857028020.d7
Van Leeuwen, T. 2008. Discourse and Practice. New tools for Critical Discourse Analysis. Oxford:
Wodak, R. 2008a. The contribution of critical linguistics to the analysis of discriminatory
prejudices and stereotypes in the language of politics. In R. Wodak and V. Koller (eds.),
Handbook of Communication in the Public Sphere. New York/Berlin: Mouton de Gruyter,
pp.291316.
Wodak, R. 2008b. Introduction: Discourse studies important concepts and terms. In R.Wodak
and M. Krzyanowski (eds.), Qualitative Discourse Analysis for the Social Sciences. Basingstoke: Palgrave Macmillan, pp. 129.
Wodak, R. 2009. The Discourse of Politics in Action, New York: Palgrave Macmillan. DOI:
10.1057/9780230233683
268 Michael S. Boyd
Wodak, R. and P.A. Chilton. 2007. A New Agenda in (Critical) Discourse Analysis: Theory, Methodology, and Interdisciplinarity, Philadelphia/Amsterdam: John Benjamins.
Wodak, R. and R. De Cillia. 2007. Commemorating the past: The discursive construction of
official narratives about the rebirth of the Second Austrian Republic. Discourse & Communication, 1, pp. 315341. DOI: 10.1177/1750481307079206
Wodak, R. and M. Meyer. 2009. Critical Discourse Analysis: History, agenda, theory, and methodology. In R. Wodak and M. Meyer (eds.), Methods of Critical Discourse Analysis. 2nd
Revised ed. London: Sage, pp. 133.
Wodak, R. and S. Wright. 2006. The European Union in cyberspace. Journal of Language and
Politics 5, pp. 251275. DOI: 10.1075/jlp.5.2.07wod
part iii
Converging methods
Introduction
Alan Cienki
Vrije Universiteit/VU University Amsterdam
The chapters in the previous two parts of this volume reveal that there are some
fundamental gaps between quantitative and qualitative methods of text analysis. These seem to concern differences in the acceptance of degrees of objectivity
between analyzing texts as data and language as social practice. It appears that
analyzing texts as data or texts as constituting social reality requires standards that
stand in the way of actually integrating the types of methods used in these two
types of approaches. The chapters in this part present two quite different forms
of combining methods in order to ascertain political parties positions, yet they
both argue that the resulting hybrid approach yields richer results than the simple sum of the individual research methods would. Their aim is to bridge the
gap between rigidity of quantificational methodology and the malleability of language use in practice. In each case, one method used provides a partial solution
that also raises questions, and one or more other methods employed provide(s)
insights which can address those questions, allowing for a more comprehensive
analysis on the whole. Krouwel and Walls chapter (13) outlines the procedures
used to develop a particular type of online Vote Advice Application (VAA). The
converging methods here include the analysis of political texts of different sorts
by a research team to position the political parties concerned and then consultation with the political parties themselves about their interpretations made by the
team. The chapter by Van Elfrinkhof, Maks and Kaal (Chapter 14) concerns a
convergence of approaches from political science, computational linguistics and
cognitive and critical discourse analysis. A key difference between the approaches
that are integrated here is how the words are handled as data: as elements which
can be treated as loose items in bulk or as units which must be considered within
the particular contexts in which they are situated. Ultimately, these chapters themselves are convergent with each other, in that the Vote Compass VAA described
in Chapter 13, and specifically the one designed for the Dutch national parties,
provides the gold standard for testing the hybrid of methods used in Chapter 14.
In turn, the linguistic analysis of the kinds of words used most frequently by the
different parties in the manifestos analysed in Chapter 14 provides insights into
272 Alan Cienki
what the axes in the Vote Compass mean in terms of how party positions play out
in the language they use.
An interesting difference between the chapters is what they reveal about the
analysis of the explicit and the implicit in texts. For example, the chapter of Van
Elfrinkhof et al. shows different ways in whichthe positioning of political texts
(or sets of texts) along a dimension can involve using relative word frequencies.
This is quite different from the method used in Krouwel and Walls chapter, in
which positioning is arrived at through the interpretations of the relevant texts
by the researchers or the parties that produced the texts. While word frequency
analyses rely on explicit mention of concepts that can serve as evidence of parties worldviews and policies, the reading and interpretation of texts as wholes by
humans can yield insight into what may be implicit in them: such reading draws
on the readers background knowledge and beliefs that has not been mentioned
in the text. The use of certain key words by the writer of a political text can invoke
larger frames of reference in the mind of the informed reader through metonymic
links (Lakoff 1987), whereby some part mentioned stands for a larger whole (what
some in US political circles call a dog whistle heard only by the intended audience). For example, some right-wing American politicians might use the term
big government to induce an inference from their constituents that they are
against higher taxes (since in their worldview, taxes support a governmental infrastructure which they consider to be excessive and bloated). The approaches in
these two chapters therefore not only highlight the value of using hybrid means of
analysing texts to estimate political positions; they also point to the need for further integration of text-explicit (e.g., quantitative) and text-implicit (contextually
interpretive) methods of analysis of political language use. In this endeavour, not
only discourse analytic methods, but also approaches from fields such as cognitive
linguistics are proving useful for revealing how readers, hearers and viewers are
making sense for themselves of the language of political and policy discourse (see,
for example, Cienki and Yanow 2013).
A key theme running through these chapters is that the process of rendering political parties principles and goals (abstract notions) in visual terms for
analysis and/or use in a VAA inherently involves a metaphor: those abstract concepts become objectified and spatialized onto axes (showing a range, such as from
progressive to conservative). This metaphor likely seems unproblematic and even
invisible to most readers of this volume, which is not surprising, given its deeply
entrenched status in our language about politics, constituting part of the political
metaphors we live by ( la Lakoff and Johnson 1980). We refer to political positions
(as if they were spatial locations) and the left-right dimension, which inherently
frames our view of parties along a lateral axis. These spatial metaphors have historical bases which have become conventionalized, such as in the physical, historical
Part III. Introduction 273
origins of the political left and right in the French parliament in Versailles in the
late 18th century, with the supporters of the king seated on the right side when
viewed from the front of the hall, and those against the monarchy and in favour
of democratic change on the left (Laponce 1981). What we see in the research discussed in the following chapters is an extension of this principle of spatial mapping
in nuanced ways, giving rise to two-dimensional spaces. It is important to bear the
metaphorical bases of these spatial axes in mind, however, so as not to lose sight
of the fact that this reification does not itself constitute an immutable truth. The
following chapters provide insights both into the ways in which researchers try to
estimate these positions (sometimes with divergent results due to differences in
methods used) and the kinds of interpretive processes that are involved in doing
this placement.
Finally, note that the objectification involved in locating the parties on a spatial grid can give a misleading impression that these positions are static and stable.
However, we must not lose sight of the fact that parties have varying degrees of
dynamicity in their stances on issues. While any one analysis can present a snapshot of the parties ideas and policies at a particular point in time, analyses and
VAAs made from data in different years in the same country can also provide
valuable information for researchers on the dynamics of party politics.
References
Cienki, A. and D. Yanow. 2013. Why metaphors and other tropes? Linguistic approaches to
analyzing policies and the political. Special issue: Linguistic approaches to analyzing policies and the political. Journal of International Relations and Development 16, pp. 167176.
DOI: 10.1057/jird.2012.28
Lakoff, G. 1987. Women, Fire, and Dangerous Things: What Categories Reveal about the Mind.
Chicago: University of Chicago Press. DOI: 10.7208/chicago/9780226471013.001.0001
Lakoff, G. and M. Johnson. 1980. Metaphors We Live By. Chicago: University of Chicago Press.
Laponce, J.A. 1981. Left and Right: The Topography of Political Perceptions. Toronto: University
of Toronto Press.
chapter 13
From text to the construction

ofpoliticalparty landscapes
A hybrid methodology developed
forVotingAdviceApplications
Andr Krouwel and Matthew Wall
VU University Amsterdam/Swansea University
This chapter describes a new, hybrid methodology to measure and code party
positions on salient political issues. This new method combines expert-judgements based on text analysis with self-placement of parties. The resultant reliable party positions can be used to construct online applications that help voters
to decide which party is closest to their own political preferences. The chapter
presents the methodology used to identify salient issues and to frame these
issues into statements to which users of the online tools are asked to respond.
Furthermore, consideration is given to how relevant party documentation is
identified and how these textual sources are used to calibrate political parties on
the issues. Finally, the chapter discusses the methodological problems of aggregating voters opinions into an easy-to-understand output.
Introduction
This chapter describes a new party ideological profiling method that was developed
at VU University Amsterdam in 2006 for legislative elections in The Netherlands.
Parties were asked to place themselves on 30 to 36 issues, while experts handcoded these parties on the same issues, using a multitude of textual resources
(manifestos, websites, policy documents, speeches, debates, etc.). Subsequently,
the expert hand-coded placements were compared with parties self-positioning,
after which the experts engaged in a dialogue with the parties to clarify their position on issues where discrepancies occurred. This hybrid method of expert coding through text analysis and party self-placement was developed to create a new
generation of online applications, designed to map voters policy preferences onto
party policy positions. These online applications are typically referred to as Voter
276 Andr Krouwel and Matthew Wall
Advice Applications (VAAs), although some developers prefer the term Voter
Engagement Applications. Designers of such sites face an applied text to position challenge they derive quantitative, reliable, and transparent measures of
the political positions of parties and candidates on the basis of verbal and textual
artifacts. These artifacts include party manifestos, websites, policy documents,
press releases and speeches made by party representatives and leaders. All of these
data provide clues as to the policy positions of political actors.
This chapter outlines the challenges and decisions implicit in the construction
of such a VAA site, focusing on the estimation of parties ideological positions
on the basis of party-generated textual data. For an in-depth discussion of issues
surrounding the presentation of results and transparency of the procedure, see
Krouwel et al. (2012). We describe the steps involved in this calibration process of
parties and candidates on salient issues on the basis of official party documentation, outlining the substantive and methodological decisions that are made when
constructing Vote Compass (Kieskompas) sites in European, North American
and northern African political systems. VAA sites are live during political campaigns, and given that their core function is the provision of voting advice, it is
unsurprising that their characterizations of parties political positions are subject
to intense scrutiny and often bitter contestation. The convergence of methods
identified by Cienki in his introduction to this section represents an attempt to
balance these conflicting considerations. Accusations of bias are common, meaning that total transparency of coding procedures and deep engagement with the
political parties themselves are key requisites of creating a trustworthy VAA site.
On the other hand, given emerging evidence that VAA advice is politically influential (Wall et al. 2012), parties face substantial incentives to seek to game VAA
developers, by presenting positions designed to maximize the number of recommendations that they receive. As such, this chapter provides a practically oriented
insight into the challenge of converting text to position, which complements the
academic debates and analyses developed in this volume.
The main focus of this chapter is to describe the conceptual design and
methodology used for measuring party policy positions for VAA sites based on
textual sources. We outline the shortcomings of the most influential existing
approaches and discuss the hybrid methodology that was developed to code parties in Kieskompas VAA projects since 2006. We explain how Kieskompas projects
employ an iterative and plural approach to selecting the issues that matter in each
election. Subsequently, this chapter describes the hybrid and interactive party calibration method developed by Kieskompas, discussing how it seeks to simultaneously minimize the potential for bias in the party placement process and maximize
transparency for site users. We particularly focus on aspects of the selection and
analysis of party texts upon which coding decisions are based, as well as on the
Chapter 13. From text to the construction ofpoliticalparty landscapes 277
final site output that allows users to compare their own political preferences with
the positions of parties. We discuss site-design innovations that provide users with
information about the type of analysis they receive, either as an overall outcome
or an issue-by-issue or party-by-party comparison. Our chapter concludes with a
discussion of likely future developments in VAA-site execution, emphasizing the
importance of maintaining high standards of rigor and transparency in positioning parties and candidates on issues to ensure that future VAA sites provide nonbiased, objective, and fact-based advice to their users (see also Geminis 2012).
The complex nature of modern politics and the birth of VAA websites
One may rightly ask why VAA designers would invest much effort in turning
political textual data into estimates of party positions. The answer is that academic
VAA projects aim to provide the public with independent, transparent, and high
quality information on party policy. This is important because, in liberal representative democracies, one of the ways in which political parties or candidates
compete for votes is on the basis of policy proposals. During election campaigns
parties and candidates emphasise certain problems faced by either society as a
whole or by an identifiable group within society and promote their partys solutions to these problems. Such political arguments are mostly communicated in
spoken and written language, and voters need to be able to differentiate between
parties on the basis of the policy positions that they advocate. Once in office, government parties typically seek to implement the policies they advocated during
the election campaign, on the basis that they have received a popular mandate to
do so as a result of their election. As such, the capacity of the electorate to make
meaningful distinctions between competing party policy positions is essential to
the functioning of modern democracies, and voters can only make meaningful
distinctions on the basis of reliable political information. The core goal of VAA
sites is to provide voters with such information in a structured manner. Various
studies into the use of VAAs show that a growing number of voters are consulting
VAAs to help them decide on which party to vote for (Wall et al. 2012). Developers
therefore seem to have a significant public responsibility to aspire to the highest
standards of transparency, objectivity and reliability in the construction of VAAs.
One must also view the function of VAA sites in the light of confusing and
unstable patterns of political competition and the way politics is broadcast through
mainstream media. Distinguishing issue stances and party ideology to help voters
understand party competition in the 21st century is important, because an attenuation of profound ideological differences between traditional political parties, and
a radical decline in affective partisanship among voters have both taken place
across established democracies since the late 1950s (cf. Dalton and Wattenberg
2000). These trends mean that modern voters often struggle to define their own
ideological orientations as well as being uncertain of the policy stances of the
parties who compete for their votes. At the same time, research shows that issues
and policy considerations have become more important for voters while in the
same period they have been de-emphasized by political parties in Europe and
beyond (Franklin 2003; Thomassen et al. 2000). The voters task is made even more
difficult by the proliferation of new parties across the European party systems,
and the corresponding decline in the popularity of traditional parties, which has
transformed the national sphere in many countries into a fragmented and multifaceted political landscape. This growth in the numbers of electorally viable parties
leaves voters facing a problem of information overload. All of these factors mean
that modern Europes voters face a challenging decision: which party should I
vote for?, with few reliable transparent resources on which to base their decision.
Given these complexities faced by modern voters, it is not surprising that
websites have been developed in many countries to help voters pick the party that
is closest to their own policy preferences, and that they have been particularly
popular in countries with highly fragmented party systems (Walgrave et al. 2008).
In the Western world, VAA sites have become popular with voters for instance
the Dutch StemWijzer and Kieskompas sites generated 4.7 and 3.4 million advices
respectively in the 2006 campaign. The 2009 German Wahl-O-Mat site reached
over 6 million users and the 2010 Canadian Vote Compass site, developed by
Kieskompas, was visited by over 4 million users. Some VAAs have gone international and produce websites in various countries. Kieskompas sites, for example, have reached millions of voters during election campaigns in Sweden, Israel,
Belgium, Portugal, Turkey, and the USA and recently in Latin America and several
of the Arab Spring countries of northern Africa.
Locating parties or candidates in a common political space

However, a major problem for VAA designers is that there is no current consensus among political scientists as to how salient issues can be identified, how to
extract party positions from textual data sources and how voters opinions should
be compared to parties policy positions. One of the most contested issues is how
to select the most salient topics in an election campaign and then frame the questions in such a manner that they can be used to elicit voter and party positions
on these issues. Another major bone of contention is whether all issues should be
treated separately, meaning that voter-party correspondence is simply the sum of
all issues or that specific issue questions can be combined to identify deeper-lying
ideological opinion structures of both parties and voters. Most importantly, VAA
designers have to assign political positions to parties on the issues that they foreground. There are multiple competing methods for measuring party positions
within the field of political science. Locating parties or candidates in some form
of common political space, often in left-right terms, generates a debate about the
most important ideological or policy dimensions of that space. In most countries
there is no consensus whether the political space is one-dimensional or multidimensional, which cleavage dimensions are most salient and which issues belong
to which profound ideological dimension (see Wagner and Ruusuvirta 2012).
Political scientists devote considerable time, effort and expense to the study
of parties policy stances and they have developed several methods to determine
the ideological or policy positions adopted by parties. Examples of such methods include: expert surveys, surveys of placement of parties by voters, surveys of
political parties, roll-call behaviour of politicians and content analysis of party
manifestos (for a detailed discussion, see Benoit and Laver 2006:123153). Expertsurveys, elite-surveys (for example of MPs) or mass surveys to establish party positions suffer from the fact that no real position of the parties is measured, but only
the perception of experts, voters or party representatives. In addition, such party
placements based on perceptions are bound to be contaminated by a certain level
of bias and evaluations of past behaviour, rather than an estimation of the current
position. Thus, such perceptions may underestimate changes in the issue positions of candidates and parties. Another method by which party positions have
been generated is by text analysis of formal party documentation, such as election
manifestos, party-leaders speeches, policy documents, party websites, parliamentary debates, party propaganda and campaign material, public debates and media
content as discussed in this volume (see also Electoral Studies 26:1, 2007). The
most elaborate content analysis project of party manifestos based on the interpretation of human coders is the Comparative Manifesto Project (CMP) (see
Klingemann et al. 2006). However, one major problem with the CMP approach is
that they simply count the number of sentences devoted to a limited, and predetermined set of issues and are basically measuring salience, not actual issue positions.
Moreover, the CMP methodology generates strong centripetal estimates of party
positions compared to hand coding (Krouwel 2012:193205). Such large text corpora have more recently been subjected to analysis via automated methods such
as Wordscores and WordFish, which focus on the frequency of particular words.
As explained in this volume by Collette and Ptry and Van Elfrinkhof et al., with
both of these automated methods it is possible to position parties on a political
dimension, based on the relative frequency of specific words in the selected texts.
The Kieskompas method presents a step beyond these established methods by
combining expert judgements with self-placement of parties. Improving further
on expert surveys, the Kieskompas method fully documents and justifies each
positioning of parties or candidates with text snippets from official party documentation. Improving further on previous methods political parties are locked
into an interactive process in which the parties are also asked to position themselves on the issues. The expert placements are then compared to the auto-positioning of the party. When the party disagrees with the expert coder the party is
asked to provide alternative documentation that could justify its auto-placement.
Naturally, the expert-coder also discloses to the parties the text snippets on which
the expert placement was based and provides an explanation of their judgement.
The outcome of this interaction is either a revision of the expert judgement, or the
party accepts the coders judgement and modifies its self-placement in the direction of the expert-placement. Unlike traditional expert-surveys, the Kieskompas
method leads to a database of party positions on salient issues in which (a) each
position attributed to the party is justified by a reference and hyperlink to a text
snippet from an authoritative party document and (b) each position is authorised
by the parties themselves (with very few exceptions where disputes remain). In the
next section, the Kieskompas method is described in more detail.
The Kieskompas methodology
The Kieskompas approach to party positioning involves six key steps:
1. Identifying the salient issues in a specific election year, i.e., selecting the issues
on which political parties or candidates will be positioned.
2. Selecting authoritative textual data sources party platforms, policy documents, websites and campaign material from which party positions will be
extracted, as well as creating a hierarchical order among different types of
documents.
3. Framing propositions about these salient issues, that give ideological direction to the proposition and determines its loading on one of the deeper cleavage dimensions that demarcate the political landscape.
4. Selecting the relevant political parties or candidates to be included in
Kieskompas and which are calibrated on salient issues by analysing their official policy documentation.
5. Combining hand-coded expert placements on issues with self-placements of
the political parties or candidates.
6. Comparing voter opinions to issue positions of parties or candidates, i.e., summarising the voter-party comparison rules that drive the sites output.
In order to make these crucial decisions, Kieskompas always collaborates with

a team of (native) country experts, well versed in election surveys and electoral
politics. We aim to have broad academic teams, often four to eight experts to
avoid any one actor dominating these processes, which reduces potentially biased
outputs. In addition, we form a large coding team, including young scholars who
have not been involved in the original selecting and framing of the issues and
thus are not contaminated by the debate on how to frame issues in order to distinguish the parties or candidates from one another. This team is always identified
on the website. Another safeguard against bias and an important innovation is
that unlike its predecessors Kieskompas is completely transparent with regard
to source texts and coding. In order to maximise transparency, all text extracts
underpinning the party calibrations are made available to each user with a simple
mouse-click, allowing voters to also enter into a debate on the position of the parties on these issues.
Below we describe the procedures followed in this multifaceted process of
calibrating the most relevant parties on salient issues. This method of positioning
parties in a policy space was first developed in 2006 for the national election in
The Netherlands (see Krouwel and Fiers 2008) and was subsequently applied in
other party positioning projects (see Arian et al. 2010; Trechsel and Mair 2011).
1. Issue identification and selection: Determining whats
at stake in the election
The selection of the issues to be included in the VAA is the most crucial choice to
be made in the construction of the application (Walgrave 2008). Although methods differ substantially most party profilers start by drawing up a list of relevant
issues, and subsequently narrow down this list to a final selection (Deschouwer
and Nuytemans 2005; Krouwel and Fiers 2008:710; Laros 2007; van Praag 2007).
In order to collect all possible issues, each party-positioning project at Kieskompas
starts by selecting the relevant texts, including websites, through which parties
communicate their policy stances (see below). These relevant campaign issues are
then analysed through a computer-assisted saliency analysis of these texts (see
Grijzenhout, Marx and Jijkoun, this volume). This frequency list, generated by
computer analysis, is complemented with two further issue lists. One of them is
produced independently by the country experts and is based on their existing deep
knowledge of the subtleties of the countrys politics and party competition. They
carefully read the party programmes and other relevant text sources to identify
salient issues, not simply by their frequency, but on a scale of urgency or strength
of the formulation of the partys position on issues. This approach prevents the
exclusion of issues that are mentioned less frequently but are phrased in very strong
terms, and are therefore important for the positioning of a party. The experts also
identify issues that are relevant for the historical or current pattern of party competition. When, for example, a party has recently split over policy disagreement,
then that indicates that the issue causing the split is important enough to warrant
inclusion. Also the emergence of new parties, indicating that there are issues that
were insufficiently represented by established political parties, makes their issues
relevant. A third frequency list is compiled by a group of journalists, most of them
from the media partners we work with to promote the site. They are asked what
they perceive to be the most relevant issues in the upcoming elections, based on
their observation of the political debate on a day-to-day basis. In addition, we ask
these journalists to indicate the core of the political dispute at stake, allowing us
to frame the proposition properly. The involvement of numerous experts combined with computer-assisted content analysis is designed to minimise bias in the
issue selection. The combination of computer-assisted text analysis of party documentation and expert opinion in selecting salient issues is a substantive empirical
improvement over a priori selection of issues and bipolar policy dimensions.
All three issue lists are then compared and the most frequently mentioned
and the most strongly emphasized issues are included in a list of roughly 35 to 55
issues. Although normally around 30 issues are included, parties are positioned
on more issues than necessary to be able to exclude issues on which parties cluster
too much. The selected issues need to conform to several requirements. First, they
should be salient for political competition between parties as well as meaningful
in the political debate. Second, the selected issues should address a wide range of
relevant policy fields in order to prevent a bias towards specific parties that own
certain issues. Third, the issues must be able to differentiate between parties. Issues
on which parties broadly have shared goals, such as security, should be avoided.
These valence issues or emtpy signifiers touch on such broad and primary issues
that parties can only differ on the ways to achieve the same goal (cf. Walgrave
2008, and Montesano Montessori).
2. Selecting and analysing relevant party documents
The most stable cross-time source for party positions is the official documentation
parties themselves produce in order to communicate their policy positions (Budge
2000; Laver and Garry 2000). The main source for party positioning in an election year is the manifesto that is especially written for the upcoming election (Van
Elfrinkhof et al., this volume). However, manifestos also have several drawbacks
for VAA designers, and political scientists in general: they are often strategic (i.e.,
designed to please large groups of voters, rather than explain a partys true policy
stances); they differ in terms of length and topics addressed across parties (important issues are not always addressed in all manifestos as parties de-emphasise, or
background, issues that can hurt them electorally); and the manifestos of government parties systematically differ from those of non-government parties, the latter being less constrained by recent track records and possible future (coalition)
government participation (Marks et al. 2006; Marks et al. 2007:2627). In addition,
new parties may not have a policy position on each issue, or may focus heavily on a
single issue. Thus, party manifestos are important, but they are not the sole source
of the stance of parties on issues. In modern campaigning, many parties use their
website to provide citizens and journalists with information about their position
on issues. Party websites are particularly important to keep up with new topics
that crop up in an election campaign, or when the mood changes due to economic
crisis, natural disasters, internal unrest or international conflict. In such events,
parties issue press releases and put reactions on their website. Websites are also
important to provide up-to-date justifications and motivations for policies that may
be ambiguous, unclear or too generally formulated in the manifesto. Most parties
have more extensive information on their website, including statements by leaders, press releases, amendments accepted at party conventions and even full policy
documents on specific issues. Media reactions of party leaders and representatives
can also be a source for party positioning on issues. These can be carefully planned
speeches that are timed and targeted at specific voter groups or to get media attention. In addition, debates on radio and TV often serve to introduce new or modified positions and often clarify stances taken. During debates, when probed by the
interviewer or opponents, party representatives often elucidate their position on
issues in order to distinguish themselves from their competitors. However, the relative importance of each mode of party communication differs per country.
Academic teams in each country need to rank-order the various sources from
which policy positions can be extracted in terms of their political weight. This
hierarchy depends on national and party norms: in some systems a leaders statement will supersede a party manifesto position, in others it will not. While in
traditional mass parties the manifesto will outweigh any policy statement made
by an individual party representative, the leader of a populist leader-centred party
may override the official party programme. The final decision to sort the most
authoritative source is left to the country experts. It may occur that two sources
have identical weight, in which case, the most recent source is given precedence;
so current manifestos outweigh previous manifestos, recent speeches by the leader
outweigh those made in the past, and so on. This hierarchy of documents is then
integrated into the coding template. Before the calibration of parties on issues
can start, two more elements are needed: the propositions and the parties to be
included in the VAA.
3. Framing and scaling the issue propositions

Once the most salient issues have been selected, they need to be carefully framed
into propositions that trigger taking position on a scale that correlates with political dimensions of party competition. As Eleveld (this volume) explains in her
discussion of policy narratives, policy issues can be framed in such a way as
to emphasize certain aspects and ignore others. Simply put, it is very difficult
to phrase a neutral political statement or ask a neutral political question. To
avoid bias towards one ideological orientation or one party or candidate, the
Kieskompas method recognises that, in order for the set of questions to be balanced, propositions need to be framed in different political directions. Through
this variation in framing, each of the propositions is thus scaled towards one of
the four poles of the political landscape. The entire set of propositional statements
must be balanced in terms of subjective framing for the Dutch Kieskompas
sites, this means that a quarter of the questions have rightist framings, a quarter
leftist, a quarter progressive and a quarter are formulated in the conservative
direction. For example, a rightist framed proposition on socio-economic policy
in the 2011 Canadian Kieskompas ran as follows:

(1) When there is an economic problem, government spending usually makes it

worse.
An example of a left-framed item on the same socio-economic issue cluster is:

(2) It should be easier to qualify for Employment Insurance
The total set of propositions is also grouped into broader issue clusters or topics,
such as healthcare, environmental policy, welfare state, foreign policy, etc. Within
a broader topic, several propositions are thus framed in a variety of ideological
directions in order to reduce the danger of acquiescence bias, pushing the voter
towards a certain response (Evans et al. 1996). Also, clustering multiple propositions reduces the effect of the selection of specific issues and allows for more subtle
levels of differentiation between parties, particularly when they are in the same
political block or in the crowded political centre. Both the selection and framing of
issue propositions are vital to the functionality and fairness of the tool however,
we must acknowledge that issue selection and proposition framing are human
work. It is done in the most rigorous manner possible, but it remains a qualitative
and interpretive process. This means that a different team may select alternative
issues and framings to capture policy competition. Walgrave et al. (2009) point
out, for instance, that the selection of issue propositions affects site output dramatically. However, bias can be minimised by using a range of diverse sources and
experts and balancing the results in issue selection and framing. In a final step,
survey-experts are asked to check the formulation of these propositions to prevent

design pitfalls of proposition formulation (van den Berg et al. 2002). Once more,
the experts and journalists can comment on what issues are missing from the list,
how the issue proposition is framed and if the proposition indeed taps into the
core of the matter.
Another aspect of the framing dilemma is the provision of answer categories. According to the method adopted by Kieskompas, propositions need to be
formulated in a way that stimulates a reaction that can be measured on a 5-point
Likert scale. One can strongly disagree, somewhat disagree, neither agree nor
disagree, somewhat agree, or strongly agree with the propositions. The answer
categories can vary, so that propositions can also be framed towards a five-point
scale ranging from much more to much less, from much lower to much higher,
et cetera. This represents a major source of differentiation from many alternative
VAA designs, which employ dichotomous answer frames.
4. Selection of relevant parties: Whos in and whos out?
Ideally, all parties that participate in the election should be included in a party
profiler. In some versions we have been able to include all parties. However, in
multi-party countries such as The Netherlands with large numbers of parties registering for the election, a selection seems appropriate to filter out the micro parties
that do not stand a chance of gaining parliamentary representation. This filtering
improves the effectiveness of the profiler by reducing complexity, yet parties are
excluded. In order to avoid exclusion of electorally relevant parties we minimally
use two key criteria for inclusion: (1) all parties with one or more seats in parliament that enter the elections are automatically included; (2) parties that do not
have any seats in the outgoing parliament but which consistently poll at least one
seat in a number of opinion polls are included in the analyses. In candidate-based
elections such as the US presidential elections (see www.electoralcompass.com)
candidates were included that consistently ended up in opinion polls as serious
candidates with a genuine chance of winning the partys nomination.
In countries with no electoral history or reliable opinion polling, and where a
wide variety of new parties exist, criteria for party selection need to be adjusted.
Such criteria are, for example, having formal recognition and legal status, fielding
candidates in a sufficiently large number of constituencies to be nationally relevant
and being considered a viable contender by experts. Thus, we exclude likely unsuccessful micro-parties. So far, our academic teams have been able to identify almost
all the relevant parties or candidates, with only one minor exception in the very
first Kieskompas for the 2006 Dutch parliamentary elections. The new Animal
Rights Party was not included but gained two seats in parliament.
5. Hand-coded party positioning from political text-analysis

The Kieskompas calibration method is primarily based on hand coding.
Automated coding methods of political texts by computer programs do exist
(Benoit and Laver 2003, 2007) and the appeal of such an approach lies in the
removal of the human costs involved in hand-coding as well as the potential for
human bias in interpretation. Yet there are major methodological problems with
currently available automated coding approaches. Fundamentally, as Collette
and Ptry (this volume) note, such methods fail to take the grammatical structure and meaning of the sentences of which texts are composed into account,
and they find that Wordfish provides particularly unreliable results. Thus, these
methods do not tap into the connotation of words in their context crucial in
political communication and they chunk texts into words, negating semantic,
grammatical and discursive structures and frames that give meaning. Moreover,
automatic text analyses are incapable of distinguishing positive or negative
direction in a text, which is crucial for identifying relative party positions on
issues. Collete and Ptry also find that both language (French versus English)
as well as document length influences automated coding results. Analysing the
Canadian political landscape, they find that with automated text coding the
Canadian Liberals and Conservatives switch position, contradicting the substantial ideological distance between them that is found through hand-coded
positioning (www.votecompass.ca).
Wordscores and Wordfish analyses can yield promising results, yet parties do
not always align in the same order along ideological dimensions when compared
across languages. Furthermore, Van Elfrinkhof et al. (this volume) indicate that
Wordscore does not perform well in terms of its ability to situate parties on the
progressive-conservative (or, GAL/TAN) dimension, although it does perform
well when situating parties on the left-right economic dimension of competition.
While Van Elfrinkhof et al.s analysis points towards the potential of automated
coding to complement qualitative/human coding, this development is still in its
experimental phase, and is prone to substantive error, as described in Chapter 14
(Van Elfrinkhof et al., this volume). Given the intense scrutiny placed on party
codings in the construction of VAA sites, such aberrations, which defy common
political sense (such as the automated placement of the PVV produced in Van
Elfrinkhof et al.s analysis) are an unacceptable risk for VAA developers. Until
some severe technical and methodological obstacles are overcome, Kieskompas
will continue to use human coders. Considering the importance of VAAs we have
to accept that hand coding is an interpretive approach, which is very labourintensive, expensive to replicate or change and sensitive to individual errors of
interpretation and low inter-coder reliability (see Budge et al. 1987; Collette and
Ptry, this volume; Klingemann et al. 1994; Laver and Garry 2000; Laver and
Hunt 1992).
For Kieskompas applications, those who draft the propositions and calibrate
the parties are selected on their academic background and relevant knowledge of a
countrys party system. Propositions are framed so that they can be answered with
labelled five-point item Likert scales (Revilla et al. 2009). This format of answer
categories was adopted after careful examination of another Dutch-designed
VAA (Stemwijzer), which employed binary answer categories (agree versus disagree, as well as a dont know). Until 2003, Stemwijzer applied the binary system,
but decided to include an intermediary neutral position, after severe criticism
that it denied parties and voters the opportunity to adopt a nuanced position.
Groot (2003:2324) has shown that binary answer categories resulted in incorrect
(self-) positioning of parties on multiple propositions, which distorted the voting advices given. Dichotomous answer categories also reduce the discriminatory
power between parties belonging to a similar political orientation and between
centre parties (Arendsen 2003; Groot 2003:2427; Kleinnijenhuis and Krouwel
2007, 2011). Based on these findings, Kieskompas opted for Likert items with five
response levels, which measures attitudes in terms of level of agreement or disagreement on propositions. With Likert items, parties and users can indicate the
direction of their attitude, as well as the intensity to which they (dis)agree with the
proposition. This allows for a more nuanced differentiation between parties, even
when they belong to the same political camp. Another advantage of using Likert
items is that issue propositions can be considered items of more profound political
dimensions, such as the left-right or conservative-progressive divide.
In order to maximize inter-annotator reliability, each placement is done and
checked by multiple coders. In several Dutch versions, coders were assisted by a
specially developed text-search tool to find relevant sections of the documents
that indicate policy positions (Grijzenhout et al., this volume). Chunking texts
into paragraphs enables coders to extract multiple text snippets with the same
keywords from a single-party manifesto and across different parties manifestos.
These overviews assist coders in selecting the most appropriate text snippet(s)
and analysing the differences between snippets from different parties. Naturally,
the hierarchy of political texts is applied to select appropriate text snippets for
positioning each party on the issues. When the first source in the hierarchy does
not contain any reference to the issue concerned, annotators search in the second
level document (for example the party website), and so on. It is then decided how
this text places the party on the issue propositions. All text snippets that are used
as evidence for the calibration of parties positions are added to the Kieskompas
website and made accessible with hyperlinks, so that the positioning of parties
and candidates is completely transparent, also to the user, as shown the example from the Canadian 2011 VAA (Figure 1). In addition, Kieskompas websites
provide hyperlinks directly to the original document that was used to position
the party.
Figure 1. Transparent Coding: User interface for checking sources for party coding
on the Canadian Kieskompas site.
Notwithstanding the hybrid method applied, pure objective party calibration will
remain impossible as language production and interpretation is subjective and
ambiguous. The calibration is also complicated by other problems. Sometimes no
position can be found in official texts for some parties. If then the party does not
want to identify its stance on the issues by self-placement, the party is coded as
no opinion, for lack of a better solution. The Kieskompas method also encounters
the limitations of the assumption that parties are unitary actors. When internal
party cohesion is low and parties are internally divided into factions, it may happen that members of the same party state different policy positions. In such cases,
Kieskompas usually opts to show the different positions, including the text-snippets, in order of the official or dominant position first, followed by the deviant
positions. This often results in a calibration towards a more neutral position.
6. Self-placement and coder placement: Reaching agreement

An important innovation that the Kieskompas approach brings to party profiling methods is to combine text-based, hand-coded expert positioning with selfplacement in which the parties own assessment of their position is extracted.
Country experts select actors within each party who are authorised to express
that partys stance on a given issue. For some parties, the party leader will fill out
the self-placement questionnaire, while in others authorised party spokesmen or
campaign leaders will fill out the form. Self-placement gives parties the opportunity to challenge coding decisions. They are provided with the propositions at an
early stage so that changes in the coding can still be made. Parties are also asked
to provide appropriate text snippets from their manifestos or other official documents to substantiate and justify their self-placement. Self-placements can then be
compared with the hand-coded positioning data and well-considered adjustments
can still be made. Discrepancies between self-placement and coder positioning, as
well as the text-snippets that the coders used as a justification, are communicated
to the parties. Parties are asked to indicate what they consider to be incorrect
about the disputed expert judgement and are allowed to provide alternative textsnippets in order to substantiate their self-placement. This interactive process with
parties is generally (with a few important exceptions) constructive and leads to
clarification where policy positions were unclear. In general we find high agreement levels between expert positioning and self-placement, usually above 80 per
cent. Indeed, most of the disagreement is related to the intensity of the calibration, not of the direction. Usually, after a first round of discussion the agreementlevel increases to over 90 per cent, while multiple rounds usually result in only
very few propositions remaining disputed between the expert and the party itself.
Willingness to participate in the self-placement procedure varies across countries.
Kieskompas usually achieves compliance from the large majority of parties, while
a cross-national project during the EU elections in 2009 showed lower levels of
participation (Krouwel and Fiers 2008; Trechsel and Mair 2011).
7. Comparing voter and party positions:
Agreement or proximityapproach?
While there are many ways to compare voters and party attitudes (cf. Wall et al.
2011), most VAA models have typically fallen into two categories: agreementbased comparisons and proximity-based comparisons. Most party profilers use
the agreement approach, in which the users responses to the propositions generate a ranked list of parties, based on an additive scale of voter-party similarity.
On each issue, the user is compared to each partys position, leading to an overall
agreement score and agreement level per party. The Dutch Stemwijzer and the
Irish Votomatic VAA sites employed this type of approach (Figure 2).
Figure 2. Stemwijzer and Votomatic: Two examples of Agreement based outputs.
Proximity based profiling sites show voter positions relative to parties or candidates in a multi-dimensional policy space, whereby all the issues included are
aggregated over two dimensions (Figure 3). These dimensions differ per country
and can be adapted to the national political cleavage structure. When constructing a multi-dimensional space, Kieskompas applications use academic studies on
salient issue-dimensions. In most countries, Kieskompas utilises a material socioeconomic Left-Right dimension and a moral-cultural, non-material dimension.
The socio-economic dimension includes all issues concerning income, taxation,
government expenditure, redistribution, subsidies, welfare-state provisions, etc.
On the non-material dimension issues are aggregated that concern moral, religious or cultural topics. All propositions are assigned to one dimension only.
Figure 3. Kieskompas proximity output.
Note that the computation of averaged, or summated positions on the two dimensions depends on a priori considerations, both with respect to the question to
which of the two dimensions an issue belongs, and with respect to which end of a
dimension it leans. This means that for each issue we need to decide whether it is
material or moral, and whether the most positive score (totally agree) indicates the
left or right stance, or the progressive or conservative stance. The averaged positions
for parties and VAA users with respect to a set of specific propositions function
as the coordinates of parties and voters on the political map (Kleinnijenhuis and
Krouwel 2007). Visualization of the political landscape is thus an important part

of the tool, and this is made possible by the aggregation of multiple issues over two
underlying dimensions. The proximity approach shows the user not just whether
or not they match with each party, it also shows where they stand themselves in
terms of major structural dimensions of political competition in their country. We
discuss the process by which maps are generated in the next subsection.
While some issues may be less clearly related to one dimension or the other,
these issues may nonetheless be highly relevant to the unfolding election campaign.
We check whether each issue can be scaled to our overarching dimensions using
confirmatory factor analysis and Mokken scaling analysis. If an issue does not scale
on either dimension, it can be excluded from the algorithm used to generate the
political map and users can be encouraged to consider this issue in isolation.
7.1 Developing a multidimensional map of party competition
The formal and mathematical Kieskompas calibration procedure computes the
summated positions k Pojk of each party o and averaged positions of the voter
k Ijk on the two dimensions j1 and j2, before distances along each of the axes are
computed. The averaged positions of the parties and the user are presented graphically along the two dimensions, thus giving rise to Euclidean distances between a
voter and a party.
Ao = [ j | k Pojk k Ijk |2 ]
The site also visualises the extent to which users spatial position is made uncertain
due to ideological inconsistency (i.e., the extent to which their answers deviate
from the logical pattern anticipated by our scaling procedure) via an ellipse which
is drawn around the users issue position (see Figure 3) based on the standard
deviation of their answers on the x and y axes. This is a deliberate addition, and a
critical component of the placement of VAA users in a political space. The position of the user is not one single point, but an approximate area which will be
wide for ideologically inconsistent voters, whose answers are a mixture of left and
right, progressive and conservative, and narrow for voters whose answers point
consistently in the same ideological direction.
While both the agreement and the proximity approaches have their pros and
cons, Kieskompas nevertheless opted for the proximity approach as this reduces
the distorting effect of proposition selection. Particularly, Walgrave et al. (2009)
show that issue selection and proposition framing can dramatically affect agreement-based site output. The aggregation of responses into two deep-level ideological dimensions under the assumption that most issues can be scaled on either
the material or the moral dimension allows us to assume that deleting or adding
issues would not dramatically affect the overall positioning of either users or parties. Thus, in order to counterbalance the problematic nature of issue selection,
user and the partys positions should be aggregated in a manner that is assumed
to be robust to changes to the specific set of selected issues. However, any system
used to generate an overall result will inevitably be a simplification of the complex
realities of political competition within a country.
Conclusion
Despite their popularity, very little is actually known about the effects of different
formats of Voting Advice Applications. As Walgrave et al. point out: in sharp
contrast to their amazing popularity, to the pertinent questions about VAA outputs, and to the fierce political debate in some countries, the scientific debate about
VAAs has hardly commenced. (Walgrave et al. 2009:1162).
Only by critically comparing VAA sites, analysing different formats of voterparty comparison, question framing, and the coding of party positions will we be
able to improve web applications that inform citizens about party positions on
salient issues. While empirical evidence is still scarce, several studies show that
VAAs influence the voting behaviour of a significant portion of users (Aarts and
van der Kolk 2007; Carlson and Strandburg 2005; Ladner et al. 2010; Marschall
and Schmidt 2010; Ruusuvirta and Rosema 2009; Wall et al. 2012). In order to
avoid bias and misinformation, there is a need for academic standards for VAAs.
This chapter reflects on a novel method of VAA design, developed in 2006 and
improved over the past years and hopes to contribute to the burgeoning academic
debate around VAAs by outlining how a combination of expert-judgement, (automated) textual analysis of party political documentation, and self-placement by
political parties may result in more robust and valid estimates of party positions
than any one of these methods could achieve on their own. Bias, arbitrariness and
error are reduced by incorporating the expertise of multiple coders, broad consultation of experts in selecting issues and formulating the propositions. The variety
of official documents used as source of information and the transparency of their
origin constitute safeguards against bias. Introducing a labelled five-point Likert
scale improved our discriminatory power between centrist parties and parties on
one end of the political spectrum, compared to pre-existing VAAs.
However, the selection of relevant issues and calibration of political parties on
each of the issues is still done manually, albeit after an automated text search for
key words. While developing a hierarchy of documents is a step forward, the VoteCompass method can still not take away the ambiguity and uncertainty of some
party positions, especially when parties are internally divided. It is also extremely
difficult to code brand new parties that lack well-defined policy positions, and
solutions should be sought in text analyses for markers of more general, ideological positions. Furthermore, the aggregation of multiple textual data sources into
one single position on a five-point scale remains reductive. More should be done
to show the complexity and richness of the text snippets that underpin each calibration in a Kieskompas site. The large number of experts involved also makes the
Kieskompas method highly labour-intensive, which is expensive and impractical,
particularly with snap-elections.
Kieskompas is also developing ways to give users greater control over their
output by allowing them to determine what issues are included and excluded in
the algorithm that draws their political map. Meta-data from the Canadian and
Dutch Kieskompas shows that a substantial number of visitors indeed use these
alternative weightings to analyse their alignment with different parties. Yet, no
matter how high-tech these representations are, voters should always be able to see
an issue-by-issue analysis, treating each issue as separate and independent, as this
is the core of party voter comparisons. And they should be able to easily access the
justification of all party placements as full transparency and maximum accuracy
is crucial in order to maintain credibility and integrity.
Recently, as scholars have begun to publish and debate VAA research, there
has been a notable movement away from focusing on recommending parties to
voters, and towards seeing VAA sites as a tool for informing and encouraging voter
engagement with political issues in this conception, the real value of VAAs is their
capacity to offer voters structured information on party positions during election
campaigns, and to facilitate discussion, debate, and, ultimately to facilitate a deeper
understanding of issues and party motivation for policies among the electorate.
Within the project From Text To Political Positions researchers from
Computational Linguistics, Lexico-semantic modelling, Cognitive Linguistics,
Critical Discourse Analysis, Corpus Linguistics and Political Science combine
their strengths to develop an automated research tool for rich text-mining and
opinion-mining that will facilitate the positioning of parties (see Van Elfrinkhof et
al., this volume). Rather than focussing on differences in issue saliency, this project
tries to integrate political analysis with linguistic and discourse analytical methods
to differentiate distances in stance taking with a focus on textual affordances of
meaning making. We hope to learn from these insights to further improve and
strengthen the methodology behind our Kieskompas sites in the future.
References
Aarts, K. and H. van der Kolk. 2007. The parliamentary election in the Netherlands, 22 November 2006. Electoral Studies 26 (4), pp. 832837. DOI: 10.1016/j.electstud.2007.02.002
Arendsen, R. 2003. Partijen zonder mening winnen op Stemwijzer.nl. In Politiekdigitaal
(internet) http://www.politiek-digitaal.nl/nieuwedemocratie/partijen_zonder_mening_
winnen_op_stemwijzer.nl
Arian, A., A. Krouwel, M. Pol and R. Ventura. 2010. The Election Compass: Party profiling and
voters attitudes. In Shamir, M and Arian, A. (eds.), The Elections in Israel 2009. New Jersey:
Transaction Books, pp. 275298.
Benoit, K. and M. Laver. 2003. Estimating Irish party positions using computer wordscoring: The 2002 elections. Irish Political Studies 18(1), pp. 97107. DOI: 10.1080/
07907180312331293249
Benoit, K. and M. Laver. 2006. Party Policy in Modern Democracies. London: Routledge.
Berg, H. van den, M. Mehciz, H. Houtkoop-Steenstra and B. Holleman. 2002. Opinie-maken
of Opinie-meten. De rol van stellingvragen in markt- en opinieonderzoek. Stichting voor
Culturele Studies, Amsterdam.
Budge, I. 2000. Validating party policy positions. British Journal of Political Science 31 (1),
pp.210223.
Budge, I., D. Robertson and D. Hearl. (eds.). 1987. Ideology, Strategy and Party Change: Spatial
Analysis of Post-War Election Programmes in 19 Democracies. Cambridge: Cambridge University Press. DOI: 10.1017/CBO9780511558771
Carlson, T. and K. Strandberg. 2005. The 2004 European Parliament election on the web: Finnish actor strategies and voter responses. Information Polity 10(3), pp. 189204.
Dalton, R and M. Wattenberg (eds). 2000. Parties Without Partisans Political Change in Advanced
Industrial Democracies. Oxford: Oxford University Press.
Deschouwer, K. and M. Nuytemans. 2005. De Stemtest van de VRT: Een kijk in de keuken.
Samenleving & Politiek 12, pp. 1331.
Evans, G., A.F. Heath, and M.G. Lalljee 1996. Measuring left-right and libertarian-authoritarian
values in the British electorate. British Journal of Sociology 47(1), pp. 93112. DOI: 10.2307/
591118
Franklin, M. 2003. Voter Turnout and the Dynamics of Electoral Competition. Cambridge/New
York: Cambridge University Press.
Geminis, K. 2012. Estimating parties policy positions through voting advice applications: Some
methodological considerations. Acta Politica Online First Publication. DOI: doi: 10.1057/
ap.2012.36.
Groot, L.F.M. 2003. De Kwaliteit van Stemprogrammas en de Stemvork. Amsterdam: SISWO.
Kleinnijenhuis, J. and A.P.M. Krouwel. 2007. The nature and influence of party profiling websites. Presented at Politicologenetmaal 2007, Antwerp.
Klingemann, H., A. Volkens, J. Bara, I. Budge and M. MacDonald. 2006. Mapping Policy Preferences II. Estimates for Parties, Electors, and Governments in Eastern Europe, the European
Union and the OECD, 1990-2003. Oxford: Oxford University Press.
Klingemann, H., R.I. Hofferbert and I. Budge. 1994. Parties, Policies, and Democracy. Boulder,
CO: Westview Press.
Krouwel, A.P.M. 2012. Party Transformations in European democracies. Albany: SUNY Press.
Krouwel, A. and S. Fiers. 2008. From text to position. Text-extraction and authorisation of
party positions along dimensions of political competition. Paper presented at: Voting Advice
Applications between Charlatanism and Political Science, Antwerp.
Krouwel, A., Vitiello, T. and M. Wall. 2012. The practicalities of issuing vote advice: A new methodology for profiling and matching. International Journal of Electronic Governance. Special
Issue on Voting Advice Applications and State of the Art: Theory, Practice, and Comparative
Insights. 5 (3/4): 223243.
Ladner, A., J. Fivaz, P. Joelle. 2010. Impact of voting advice applications on voters decisionmaking. Presented at Internet, Politics, Policy 2010: An Impact Assessment. Oxford Internet
Institute (OII), University of Oxford.
Laros, J. 2007. Kiezen voor stemadvies. Over de effecten van het gebruik van stemadviesprogrammas. Unpublished MSc thesis. Amsterdam: VU University Amsterdam.
Laver, M. and B. Hunt. 1992. Policy and Party Competition. New York and London: Routledge.
Laver, M. and J. Garry. 2000. Estimating Policy Positions from Political Texts. American Journal
of Political Science 44(3), pp. 61934. DOI: 10.2307/2669268
Marks, G., L. Hooghe., M. Nelson and E. Edwards. 2006. Party competition and European
integration in the East and West. Comparative Political Studies 39(2), pp. 155-175. DOI:
10.1177/0010414005281932
Marks, G., L. Hooghe, M.R. Steenbergen, and R. Bakker. 2007. Crossvalidating data on party
positioning on European integration. Electoral Studies 26 (1), 2338. DOI: 10.1016/
Marschall, S. and C.K. Schmidt. 2010. The impact of voting indicators: The case of the German
Wahl-O-Mat. In Cederoni, L. and D. Garzia, (eds.), Voting Advice Applications in Europe.
The State of the Art (e-Book), pp. 6168.
Praag, P. van. 2007. De Stemwijzer: Hulpmiddel voor de kiezers of instrument van manipulatie?
Amsterdam: Lecture Amsterdam Academische Club, 24-05-2007.
Ruusuvirta, O. and M. Rosema. 2009. Do online vote selectors influence electoral participation
and the direction of the vote? Paper presented at European Consortium for Political Research
(ECPR) General Conference, Potsdam.
Revilla, M, W. Saris and J. Krosnick, J. 2009. Choosing the number of caterogies in agreedisagree scales. RECSM Working Paper Number 5, Universitat Pompeu Fabra - Research
and Expertise Centre for Survey Methodology, Barcelona.
Thomassen, J., K. Aarts and H. van der Kolk. 2000. Politieke Veranderingen in Nederland 1971
1998: Kiezers en de Smalle Marges van de Politiek. The Hague: SDU Uitgeverij.
Trechsel, A. and P. Mair. 2011. When parties (also) position themselves: An introduction
to the EU Profiler. Journal of Information Technology & Politics 8(1), pp. 120. DOI:
10.1080/19331681.2011.533533
Wagner M. and O. Ruusuvirta. 2012. Matching voters to parties: Voting advice applications and
models of party choice. Acta Politica, 48 (4), pp. 400422. DOI: 10.1057/ap.2011.29
Walgrave, S., M. Nuytemans and K. Pepermans. 2009. Voting aid applications and the
effect of statement selection. West European Politics 32(6), pp. 11611180. DOI:
10.1080/01402380903230637
Walgrave, S., P. van Aelst and M. Nuytemans. 2008. Do the Vote Test: The electoral effects of a
popular vote advice application at the 2004 Belgian Elections. ActaPolitica 43(1), pp. 5070.
DOI: 10.1057/palgrave.ap.5500209
Wall M., A. Krouwel and T. Vitiello. 2012. Do voters follow the recommendations of voter advice
application websites? A study of the effects of kieskompas.nl on its users vote choices
in the 2010 Dutch legislative elections. Party Politics Online First Publication DOI: DOI:
10.1177/1354068811436054.
Wall, M., A. Krouwel and J. Kleinnijenhuis. 2011. Comparing voters and parties online. The
effects of multi-attribute utility decision rule section in vote advice applications. Paper
presented at PoliticologenEtmaal, June 9-10, 2011, University of Amsterdam.
chapter 14
From text to political positions

The convergence of political, linguistic
anddiscourseanalysis
Annemarie van Elfrinkhof, Isa Maks, and Bertie Kaal
This chapter explores how three methods of political text analysis can complement each other to differentiate parties in detail. A word-frequency method
and corpus linguistic techniques are joined by critical discourse analysis in
an attempt to assess the ideological relation between election manifestos and
a coalition agreement. How does this agreement relate to the policy positions
presented in individual election manifestos and whose issues appear on the
governmental agenda? The chapter discusses the design of three levels of text
analysis applying text-as-data analysis; words-as-meaningful-data involving lexical-semantic analysis of subjectivity and words-in-context analysis for variation
in constructions of worldviews. We foundthat better results can be achieved for
party positioning in combinations of qualitative and quantitative approaches.
1. Introduction
There is a clear need for reliable estimates of party positions to inform the public and to predict election outcomes. In this chapter we explore how different
methods for the analysis of political texts can complement each other to differentiate parties using subtle distinctions. We combine a word frequency method
with a lexical-semantic analysis of modal subjectivity and a discourse analysis of
worldviews to position Dutch political parties as well as the coalition agreement
between the government parties following the 2010 national elections. The aim of
the chapter is therefore two-fold. First, in applying both quantitative and qualitative approaches we point out methodological constraints to party positioning and
show how a combination of methods could lead to a refinement of the individual
methods. Second, we compare results between coalition parties and the Coalition
Agreement to gauge the ideological influence of parties on the composition of
this agreement as evidence for further political interpretations. The Netherlands
298 Annemarie van Elfrinkhof, Isa Maks, and Bertie Kaal
is an example of a multi-party system, with over a dozen political parties vying

for a position in Parliament. Dutch Governments are traditionally coalition governments that seek common ground after national elections are held. The differences between parties are subtle and strategic: they are competitive and at
the same time they keep options open for coalition agreements. In this setting,
a fine-grained method is needed to distinguish between parties positions and
possible coalition agreements.
Concepts and words have no explicit stance, but stance is constituted in
conceptual and verbal networks that can validate abstract ideas about values,
relations and roles in dynamic processes, as Kleinnijenhuis and Van Atteveldt
mention (this volume). The challenge is to combine these three levels of communications: what do parties foreground, in which terms and how does content
and language use constitute intentions that are packaged in party programmes.
Framing seems to be the binding force in the three methods of analysis used in
this chapter. The Wordscores analysis results in typical word-frames, whereas
the linguistic and discourse analyses identify attitudinal frames of explicit and
implicit ideological stance: Most frames are defined by what they omit as well as
include, and the omissions of potential problem definitions, explanations, evaluations, and recommendations may be as critical as the inclusions in guiding the
audience (Entman 1993:54).
The idea of word frequency methods is that the relative frequency of words
reflects the policy positions of parties. Previous chapters have discussed and
shown the abilities of word frequency methods in extracting political positions
from texts. In their analysis of parliamentary debates Hirst et al. (this volume)
demonstrate that automated quantitative methods can extract competition on the
basis of texts. However, they find a governmentopposition dichotomy rather than
an ideological spectrum and therefore they cannot measure party positions, only
the role parties play at a certain moment in time. Collette and Ptrys analysis
(this volume) focuses on the influence of the structure of language on the results
of the word frequency methods, Wordscores and Wordfish. They compare the
English and French editions of Canadian election manifestos and indicate a number of grammatical differences that might be of influence in the positioning of
the parties. Both chapters conclude that word frequency methods are sensitive
to language characteristics, but they do not control for these linguistic features in
their method. In this chapter we apply the quantitative automated word frequency
method Wordscores and add knowledge of linguistic features of stance taking to
improve the performance of the method.
We focus on one main similarity across election manifestos which relates to
the purpose of the genre, namely, each party outlines their desired world to voters.
Chapter 14. From text to political positions 299
These worlds are constructed subjectively and support party stances on individual
issues. We first extract the subjective, deontic constructions from the texts and
then apply Wordscores. We find that this indeed improves the performance of the
method when positioning parties on an economic dimension. However, the results
do not improve for a progressive/conservative dimension. Relating to the chapters
by Montesano Montessori, Eleveld and Filardo Llamas (this volume) we propose
that the discursive construction of ideologically motivated worldviews forms the
argumentational backbone rationale of a party programme. It is argued that this
rationale needs to be taken into account for a valid positioning of parties on the
progressive/conservative dimension. For the critical discourse analysis we have
selected particularly meaningful text segments that outline worldviews and set the
scene for political action. An adapted version of Chiltons Discourse Space Model
(2004, 2007) is applied to find parties ground perspective to position them. The
worldview approach takes into account the importance of contextual knowledge in
the positioning of parties and adds an argumentational and conceptual dimension
to the word-frequency method.
The Vote Compass (Kieskompas) (see Krouwel and Wall, this volume) serves
as the Gold Standard against which we cross validate our results. Like the analyses
in this chapter, the Vote Compass uses election manifestos to unearth party positions. However, where the Vote Compass is mainly a qualitative method for party
positioning, we explore how the complementarity of quantitative and qualitative
methods drawn from different disciplines can improve the estimation process of
party positions.
The following section describes the Dutch political landscape to contextualise the study and account for the data selection. The design of each approach
is then described in detail, providing results of their applications to the corpus.
We first use the quantitative method Wordscores which treats words as data,
by using only relative frequencies to position parties and the coalition agreement. In a second analysis we add knowledge of semantic subjectivity markers
to the Wordscores analysis. This allows us to treat words as meaningful data.
The expectation is that the addition of semantic-categories to data improves the
performance of the Wordscores analysis. The third method is a more interpretive
discourse based analysis that posits words-in-context to account for variation in
discursive aspects of meaning constructions and stance. After the cross-validation of the three methods, the parties and the coalition agreement are positioned
in the Dutch political landscape. The chapter concludes with an evaluation and
discussion of the three methods.
1.1
Coalition formation and coalition agreements in The Netherlands
The analyses focus on the Dutch national elections of 2010. With up to twenty
parties participating in the national elections, the Netherlands has a typical multiparty system. Parties are often close to each other, both ideologically and in the
number of votes they receive, with no single party gaining a majority of the seats.
This situation has made coalition governments a necessity. Coalition agreements
have become an integral part of the process of coalition formation over the past
decades. The 2010 elections resulted in two separate coalition agreements. After
five rounds of negotiations with changing negotiation partners, the largest party,
the VVD, reached an agreement with the Christian Democrat Party (CDA) to
form a minority government with the support of the Party for Freedom (PVV).
Instead of one coalition agreement, the Rutte I Government is based on a traditional coalition agreement as well as on a tolerance agreement. The VVD and
CDA preferred this construction over a normal majority government in which
the PVV would be a full member, because of irreconcilable ideological differences on the PVVs main issues: immigration and integration. The three parties
reached a tolerance agreement that would secure a Parliamentary majority on a
large number of issues, rather than having to find majority support in parliament
on a case-by-case basis (Mller and Strm 2000). In the analysis we will consider
the relation of both documents to the coalition parties own election manifestos.
In the Netherlands, competition between political parties takes place on two
policy dimensions: an economic left-right dimension, ranging from market correcting to market liberating policy preferences, and a progressiveconservative
dimension. The latter dimension used to have a religious connotation but has
gradually changed its focus to issues of integration and immigration (Aarts and
Thomassen 2008; Kriesi et al. 2006, 2008; Marks et al. 2006). The likelihood of parties joining forces in a coalition depends on ideological similarities and the vote
share of the parties. If one assumes that political parties are policy-seekers, that
they seek agency to implement their policy preferences rather than simply seeking
the power of office, it is to be expected that coalition partners will be closely positioned ideologically on at least one of the two dimensions. When parties share policy preferences it is more likely that they will come to an agreement which reflects
their shared policy preferences. It is furthermore to be expected that the major
coalition partner, the party with most of the votes, will be the dominant partner
and be able to determine most of the agreement in its favour (Warwick 1996).
Warwick (1996) also points to the special position of the party that, although it
has not gained the most seats, has the largest increase in number of seats. They are
relative winners, and this gives them more leverage than can be expected based on
the number of seats alone, as was the case with the Party for Freedom in the 2010
elections. This context leads to the hypotheses given in Box 1, which will be tested
in order to find advantages and constraints in applying the three methods on ten
election manifestos and the two agreements.
Box 1.Hypotheses
1.Ideology: The coalition agreement is the ideological mean of the coalition parties.
2.Ideology and Size: The coalition agreement is ideologically closest to the largest
coalition party.
3.Change in Size: The party with the largest increase in the number of seats has the most
influence on the coalition agreement.
2. Data description
We used five sets of data to analyse the positions of ten political parties that succeeded in gaining seats in the Dutch Parliament following the 2010 national elections. Each of them is described below.
2.1
Election manifestos
Our main data source consists of election manifestos. The great advantage of
using texts for party positioning rather than expert surveys or opinion polls, is
that texts are stable. Texts do not change as a result of an inquiry or over time.
Election manifestos present the ideas and policies of parties simultaneously, at
a particular point in time. Manifestos are produced at regular intervals (Budge
1994) and this makes them an excellent source to study position shifts over time
(Laver and Garry 2000). Furthermore, election manifestos are authoritative
documents representative of the party community as they are usually amended
and approved by a General Assembly. Manifestos are a political-text genre that
functions to form shared beliefs, around a political organisation and to create a
coherent party identity for an epistemic community (Van Dijk 2004:9). They are
then carefully composed with the intention to communicate the partys official
position to a broad audience during an election campaign (Benoit and Laver
2006). Their primary function is to bind the party community as well as to make
the positions of the party known in a cohesive and communicatively persuasive
message. As such, manifestos are the substantive focal point for political communication during election time (Lamond 2012). The general public might not
read them, but learns about them through public appearances of politicians and
in reports by the media (Laver 2001). At the same time they serve as a guideline
for the partys politicians who can be held accountable if they divert from the
party line laid down in the manifesto (Laver 2001). Finally, election manifestos
share these characteristics across parties. In other words, election manifestos can
generally be regarded as a stable text genre with the goal of presenting parties
policy positions at a specific point in time. This makes them an excellent source
to compare party positions across time and with each other. For this study we use
the Dutch election manifestos of 2006 (EM2006) and 2010 (EM2010), together
making up the corpus EM-full.
2.2
Coalition agreement and tolerance agreement
For the application of Wordscores, Laver et al. (2003) stress that it is important
to compare like with like. The 2010 coalition agreement between the VVD and
the CDA is comparable to an election manifesto in both format and content. It
discusses policies elaborately by presenting a view on the present, ideas about the
future and policies to realize them. Therefore, the coalition agreement resulting
from the Dutch 2010 elections can be viewed as the manifesto of the coalition and
hence as a like document (Laver et al. 2003).
The tolerance agreement is a list of agreements, and agreements to disagree,
between the two coalition parties and the tolerating party (PVV). Therefore, it has
a slightly different format. We do not expect this to be a problem for the quantitative analyses. From a discourse perspective, however, the lack of argument structure is problematic. The discourse analysis focuses on text segments that explicitly
express party-specific perspectives on the state of affairs (worldviews) as their
motivation for policies and goals. Since the tolerance agreement does not contain
a comparable text segment, it is exempted from the discourse analysis.
2.3
Chapel Hill Expert Survey
In order to position parties on political dimensions with Wordscores, we use the

Chapel Hill Expert Survey (Hooghe et al. 2010) as reference scores. The 2006
Chapel Hill Expert Survey provides positions for eight Dutch political parties on
two relevant dimensions. For the election year 2010 we are interested in the positioning of ten political parties. The parties that were not included in the 2006
Chapel Hill Expert Survey are the Animal Rights Party (PvdD) and the Christian
Orthodox Party (SGP). Since we have enough variation with eight parties, we
feel confident positioning these parties on the basis of the positions of the eight
parties in 2006. The Chapel Hill Expert Survey operationalises the progressive/
conservative dimension as GAL/TAN, a scale ranging from Green, Alternative and
Libertarian (GAL) to Traditional, Authoritarian and Nationalistic (TAN) (Marks

et al. 2006). This operationalisation is used throughout this chapter. Table 1 presents the scores of the 2006 Chapel Hill Expert Survey. The parties are positioned
on a scale from 0 to 10. On the Left/Right dimension 0 is the most left position
and 10 is the most right position. On the GAL/TAN dimension 0 is the most GAL
position and 10 is the most TAN position.
Table 1. Chapel Hill Expert Survey positioning of political parties 2006.
Left/Right
GAL/TAN
2.4
SP
GL
PvdA
CU
D66
CDA
VVD
PVV
1.11
3.56
4.13
5.22
5.56
7.89
8.29
GL
D66
PvdA
VVD
SP
PVV
CDA
CU
2.4
2.5
3.7
5.1
5.22
6.57
6.7
8.4
A Dutch news and opinion corpus
A reference corpus (NO) consisting of selected news and opinion articles of 2006
was used in the words-as-meaningful-data analysis. The news and opinion texts
were randomly selected from a large database of Dutch newspapers. This ensures
the focus on national political news. Further information on this corpus follows
in Section3.2.
2.5
Vote Compass
As we see in this volume, quite a variety of methods can be used to position political parties: expert surveys, voter surveys, and party surveys are used as well as
a range of content analytic methods. Wordscores belongs to the relatively new
quantitative content analysis methods and, as such, it is not fully developed yet.
Hence, to assess the validity of the method we compare the results to a manual
content analysis method, that of the Vote Compass. The Vote Compass method is
fully explained and discussed in Chapter 13 (Krouwel and Wall, this volume). It is
a qualitative method that uses expert coders to position parties within a political
space based on election manifestos and other relevant textual sources. The positions are numerical scores on an economic left/right dimension and a progressive/
conservative dimension. The checks and balances applied by Vote Compass secure
the validity of the party positions and we have therefore chosen to use the Vote
Compass data (Figure 1 and Table 2) as a gold standard against which we can
cross-validate Wordscores and worldview results.
Table 2. Vote Compass positioning for Dutch national elections 2010.

Left/Right
GAL/TAN
SP
PvdA
GL
PvdD
CU
PVV
D66
SGP
CDA
VVD
1.76
0.88
0.82
0.82
0.58
0.23
0.35
0.76
1.29
D66
GL
SP
1.07
1.07
0.76
PvdD PvdA
0.69
0.61
VVD
CU
PVV
CDA
SGP
0.46
0.53
0.92
1.53
GAL
2
1,5
GL
SP
Left
D66
PvdD
PvdA 0,5
0
1
CU
Right
VVD
0,5
1
PVV
1,5
CDA
SGP
2
TAN
Figure 1. Visual Vote Compass positioning for the Dutch national elections of 2010.
3. From text to political positions: Three methods

for detailed manifesto analysis
3.1
Words as data
Wordscores is a quantitative text-analysis method developed by Laver et al. (2003).

Rather than coding the text on the basis of a coding schema, Wordscores treats
words purely as data to determine the position of a text on a dimension on the
basis of relative word frequency. With regard to election manifestos, the underlying
logic is that political parties do not use the same words with the same frequency.
Political parties on the right propose to cut taxes in their election manifesto.
Political parties on the left do not use the words to raise taxes, although their
plans might imply it. Left-wing parties occasionally use the word tax, but this is a
fraction of the times right-wing parties use the word (Laver and Garry 2000:625).
The word tax is considered indicative of right-wing parties. The relative frequency of words can therefore be used to differentiate between parties stances.
That is: each political party frames issues by selecting relevant words from a particular semantic field (Beigman Klebanov et al. 2008:96) that can be indicative of
party positions. Wordscores uses this formalised concept of frames to distinguish
between documents.
Wordscores uses two sets of texts: reference texts and virgin texts. From the
reference texts we know two things: (1) the words used in these texts; and (2) the
position of this text on the dimension we want to investigate. The latter knowledge is derived from external sources such as expert surveys. From the virgin
texts we know only which words it contains; we do not know their positions. By
comparing relative frequencies of words, Wordscores is able to scale the virgin
texts on the same dimension as the reference texts. This can involve any dimension a researcher is interested in: an economic left/right dimension, a pro- or
contra-dimension, or any other dimension, as long as the dimension is relevant
to both the reference texts and the virgin texts and the source of the positions of
the reference texts is reliable.
Whereas Collette and Ptry (this volume) use Wordscores to examine the ability of Wordscores to position political parties by comparing the same manifestos
produced in two languages, we aim to position agreements between parties in the
same political space as the election manifestos.
3.2
Words as meaningful data
Wordscores functions on the full text level, but texts construct subjective meaning more explicitly in some parts than in others. The main assumption is that
some parts are more crucial to constructing its message than others and that a
selection of such meaningful text segments should yield better results on stance
than the analysis of the entire text. For the analysis of words as meaningful data,
Wordscores is used again, but only on those parts of the manifestos that are more
relevant and meaningful than others.
In this study we have chosen for a lexical-semantic approach to extracting
words that indicate attitude, or deontic modality. It appears that deontic expressions (words and phrases expressing attitude) are more frequent in election manifestos compared to other text genres and are therefore considered a typical text
marker of the manifesto corpus (Table 3).
Election manifestos express parties worldviews and their desired world on

moral and ethical grounds. They propose policy measures that are deemed necessary to realise this desired world through linguistic expressions of deontic modality indicating degrees of necessity, desirability, and urgency of proposed policies
and change. Deontic language is a semantic category that indicates the degree
to which an assessor [] can commit him- or herself to the state of affairs in
terms of certain principles (Nuyts et al. 2010:17). These principles refer to moral,
ethical and socio-cultural norms, values and ideals as well as to more personal
ethical norms. However, moral obligation is not the only drive for political action
and therefore we follow Palmers (1986) approach to deontic modality that also
accounts for notions of volition and intention that express degrees of desirability.
Deontic expressions are particularly relevant to political stance in that they often
refer to a future state of affairs in a positive way, indicating what the world should
be like. Moreover, deontic expressions often refer to some kind of action towards
that ideal. In Palmers approach, deontic modality is concerned with the expression by the speaker of his attitude towards possible actions by himself and others
(Palmer 1986:121). For these reasons, we regard deontic expressions in political
text analysis as cues for moral attitude and desirability of policy measures.
This assumption is confirmed by a comparison of the election manifestos
with texts from other genres. We compare the word frequencies of EM2006 and
EM2010 (EM-full) with the word frequencies of a corpus of news and opinion
texts (NO). The texts included in the NO corpus are randomly collected from
a large database of Dutch newspapers in the year 2006 with a focus on national
political news. We use the corpus-based frequency profiling method developed by
Rayson and Garside that aims to discover features in the corpora that distinguish
one corpus from the other (2000:2): in this case, election manifestos from the
news and opinion corpus. Since the news and opinion corpus roughly covers the
same time period and the same themes as the manifesto corpus, the two corpora
have a maximal overlap of thematic words and a minimal overlap with regard
to stylistic, functional and other genre-specific words. Therefore, we expect the
comparison to highlight the words that are specific to the genre (as opposed to the
content) of election manifestos.
First, for all words in the text the part-of-speech tag and the lemma (i.e., the
canonical form) were automatically determined using the Alpino parser. Then,
lemma frequencies were measured for each lemma in the two corpora and loglikelihood statistic (LL) was calculated for those lemmata relatively more frequent
in EM-full than in NO (cf. Table3: over-represented in EM-full = +). The higher
the log-likelihood, the more significant the difference between the two frequency
scores and the more we expect the difference between the two corpora to be meaningful. If the LL of the result is greater than 10.83, the probability of the result
happening by chance is less than 0.1% (p<0,001). The list was then sorted by the
LL values and results in a ranking of LL values of which the top is presented in
Table 3.
Table 3. Top-8 ranking of saliency of words in EM-full.
Lemma
Pos
EM-full-freq NO-freq log-likelihood Over-represented

in EM-full
worden (be, become)

moeten (must)
overheid (government)
dier (animal)
zorg (health care)
dienen (must)
onderwijs (education)
duurzaam (sustainable)
verb
verb
noun
noun
noun
verb
noun
adj
8889
5157
1444
730
1002
784
918
559
2303
1551
149
10
98
50
100
8
2864.08
1353.92
961.41
775.23
684.25
631.21
594.95
591.25
+
+
+
+
+
+
+
+
NB: Pos = part of speech; EM-full-freq = frequency in election manifesto (EM-full);

NO-freq = frequency in news and opinion texts.
The top of the list shows not only genre-specific words but some traces are left of
thematic salience related to animals, (health) care, education, and sustainability.
However, there is also salience of functional words: worden (be), moeten (must),
and dienen (ought to). The over-representation of the highly deontic auxiliary
verbs dienen (ought to) and moeten (must) is particularly interesting for the purpose of party positioning as they indicate degrees of desirability/necessity.
Second, we compiled a list of conventional expressions of deontic modality
(Haeseryn et al. 1997; Palmer 2000) and used it as a set of lexical indicators to
extract sentences expressing deontic modality. Examples (1) and (2) are examples
of such sentences, extracted from the manifestos:
(1) [Deontic-moeten] Nederland moet bereid zijn humanitaire hulp te geven.
(The Netherlands must be prepared to give humanitarian aid)
(2) [Deontic/volitional-willen] We willen een open en dynamische samenleving
waarin iedereen kan
(We want an open and dynamic society in which everyone can )

We compiled a sub-corpus of these deontic sentences from each election manifesto and from the coalition agreement and the tolerance agreement. This
sub-corpus (EM-deon), which covers approximately 66% of the full manifesto
corpus, was analysed with Wordscores using EM-deon-2006 as reference corpus
and EM-deon-2010 as virgin corpus. The results are presented in the following
sections.
3.3
Comparing Wordscores results of words-as-data

to words-as-meaningful data
Before we turn to the hypotheses with regard to the position of the coalition
and tolerance agreements, we will evaluate the results of the two Wordscores
approaches and hold them against our gold standard, Vote Compass, in a quantitative as well as a qualitative way to assess their ability to position texts. Both wordsas-data and words-as-meaningful-data approaches give numeric results which
represent the position of the parties on the Left/Right and GAL/TAN dimensions.
In order to compare the positioning we have normalized the positions of all three
methods on a scale from 0 to 1. Figures 2 and 3 show the scores for Vote Compass
and present the results of the words-as-data method and the words-as-meaningful-data method. Figure 2 gives the results with regard to the Left/Right dimension
where each series of scores starts with the most left position at 0 and the most right
position is 0.9. Figure 3 shows each series of scores with positions between 0 (most
TAN position) and 1.0 (most GAL position). The best outcome is achieved by the
method whose results are closest to the gold standard scores, both with respect to
ranking the parties and the distances between the parties. In order to determine
the best outcome, we calculate the correlation between the scores of each method
and the gold standard. We use Pearsons correlation coefficient (r), which is used
as a measure of strength of linear dependence between two variables. The higher
the correlation coefficient, the better the party positions predicted by the methods
fit the actual party positions indicated by our gold standard.
1
0.9
r = 0,90
0.8
0.7
r = 0,75
0.6
Left/right
0.5
Vote compass
0.4
Words as data
0.3
0.2
0.1
VV
CU
PV
V
D6
6
SG
P
CD
A
GL
Pv
dD
SP
Pv
dA
Figure 2. A comparison of Vote Compass with words-as-data and words-as-meaningfuldata on Left/Right (2010).
1
0.9
0.8
0.7
GAL/TAN
0.6
Vote compass
0.5
Words as data
0.4
0.3
r = 0,66
0.2
0.1
r = 0,72
CU
PV
V
CD
A
SG
P
SP
Pv
dD
Pv
dA
VV
D
GL
D6
6
Figure 3. A comparison of Vote Compass with words-as-data and words-as-meaningfuldata on GAL/TAN (2010).
The results show that both words-as-data and words-as-meaningful-data have

good scores with respect to the Left/Right dimension, but the correlation coefficients (r) express that words-as-meaningful-data scores significantly better than
words-as-data with r=0.90 and r=0.75, respectively. With respect to the GAL/
TAN dimension both words-as-data and words-as-meaningful-data methods have
lower performance which is expressed by relatively low correlation coefficients,
0.66 and 0.72. Especially for words-as-meaningful-data the difference with the
performance, which is r=0.90 vs. r=0.66, on Left/Right is compelling, and suggests that the extraction of deontic modalities works better for the economic Left/
Right dimension than for GAL/TAN. Deontic modalities which are linguistically
speaking attitudinal expressions of actions, correspond to the policy measures
proposed by a party. A tentative conclusion is that the divergent performance is a
result of the prominence of policy measures for the dimensions, where the Left/
Right dimension is more starkly expressed in policy measures than GAL/TAN.
The spikes in Figures 2 and 3 indicate a different rank-order of parties on the
words-as-data and words-as-meaningful data scales compared to Vote Compass.
In Figure 2 the Social Democratic party PvdA has a more centre position on the
Left/Right dimension in the Wordscores results than Vote Compass. Furthermore,
the words-as-data analysis puts the PVV on a more extreme position than wordsas-meaningful-data and Vote Compass. Figure 3 displays a number of peaks on
the GAL/TAN dimension. Both Wordscores analyses position Green Left (GL),
Christian Union (CU) and the Party for Freedom (PVV) in a different order than
Vote Compass. The words-as-data analysis also positions the Animal Rights Party
(PvdD) differently, that is, much more TAN than Vote Compass.
The improved performance of Wordscores on the Left/Right dimension, when
applied to the meaningful parts of the election manifestos, also becomes visible
when we look at the rank order of the parties. The difference between the two
Wordscores analyses is in the position of the PVV. Whereas words-as-meaningfuldata analysis positions the PVV in the centre on the Left/Right dimension, as Vote
Compass does, the words-as-data analysis positions the PVV closer to its position
in 2006 as indicated by the Chapel Hill Expert Survey (Table 1). It also shows that
the poor performance of the words-as-meaningful-data corpus on the GAL/TAN
dimension is mainly due to the position of the PVV. It is positioned as the secondmost progressive party in the party system, whereas Vote Compass positions it as a
conservative party. We conclude that both Wordscores methods give better results
on Left/Right than they do on GAL/TAN.
3.4
Words in context
To overcome the unresolved positioning deviations for the PVV and the GAL/
TAN dimension we propose a more qualitative approach to analyse constructions
of meaning particular to a partys discourse. This meaning-based approach to text
analysis assumes that party ideologies are constructed in worldviews that cannot
be found in single words or deontic constructions, but they occur explicitly and
implicitly throughout a discourse. Although worldviews are explicit only in some
parts of a text, they function as anchors for an ideologically motivated rationale
throughout a manifesto (Kaal 2012). We therefore looked for expressions in the
data that seem to prompt assumptions about the current state-of-affairs and that
sustain a cohesive worldview from which goals and policies unfold.
The selection of text segments containing worldview frames is motivated by
the narrative structure of election manifestos and cognitive affordances of spatial representations. The theory of spatial cognition drawn upon here holds that
thought patterns for making sense of the complex world we experience have a
parallel in the organization of our spatial orientation (Levinson 2003). Spatial and
temporal location is regarded as the basic ground for evaluative thought where,
metaphorically, normative thought is mapped onto a spatial landscape. In this way,
worldview provides a reference frame with boundaries and structure for abstract
thought patterns that are anchored in real time and space. The particular affordance of such reference frames is that they can sustain a dual rationale by mapping
normative attitude (e.g., what is presumed to be good) onto empirical evidence
(a space-time reference) (Entman 1993:5556). In this way, time and space
references make it possible to express normative worldviews by bringing them

affectively closer to or more distant from the deictic centre. An example of the use
of an affective metaphor category in manifestos is comfort is closeness. This
schematic metaphor type suggests a rational structure that aligns affect with the
logic of spatial orientation. The similarity in the thought pattern connects policy
measures (to bring comfort closer) with goals (comfort is desirable) in a balanced
relationship of world-to-word-to-world direction of fit (Searle and Vanderveken
1985) that sustains the given worldview. Such schematic metaphors may not be
expressed directly (Cienki 2008); for example, in this case, they concern a way of
thinking that prompts entailments of a spatial source onto an implicit normative
target domain. The structure provided by this schematic metaphor helps to make
sense and thus provides rhetorical affordances by its appeal to shared knowledge
and experience about a shared space. In election manifestos this is the space in
which a partys programme sounds (and feels) right. We assume that ideological
party positions are expressed through these metaphorical spatial and temporal
expressions and that they are fundamental to the construction of ideologically
motivated worldviews.
The mapping of normative spatial orientation onto real space and time provides for a narrative textual structure by suggesting a measure and a coordinate
centre for direction. Time and space therefore do not only function to represent
abstract notions, but are directive, starting out from a subjective point of view (see
Herman 2003:10,165166). In this way, the spatial ground we find in manifestos
functions as a coordinate system that constitutes ideological stance and directs
political action, considering that:
An ideology is a more or less coherent set of ideas that provides the basis for
organized political action, whether this is intended to preserve, modify or overthrow the existing system of power. All ideologies therefore have the following
features:
a. They offer an account of the existing order, usually in the form of a world
view.
b. They advance a model of a desired future, a vision of the good society.
c. They explain how political change can and should be brought about how to
get from (a) to (b).
(Heywood 2007:1112)
We consider Heywoods worldview (a) as the source from which predictions for
future developments unfold and thus warrant political action (c) towards a desired
future (b). Metaphorically, the source is situated in a time and space frame.
The aim of our discourse analysis is therefore to locate the source and the
scope of worldviews to be able to differentiate party-specific ideological grounds
in which policies and goals fit. Where spatial frames set geographic boundaries to
a partys worldview, Time adds a dynamic dimension of duration and direction in

a primary metaphorical relation of time is motion along a path (Grady 1997).
Time and space are therefore regarded as anchors of ideologically motivated reference frames, or worldviews, that relate to GAL/TAN positions.
Contrary to Wordscores, the discourse analytic method did not involve reference texts but focused on discursive phenomena in the virgin texts to find subjectivity markers by abduction. The first step in this analysis was a close reading of the
2010 manifestos. This revealed that, typically, introductory paragraphs are characterised by propositions concerning us in the here and now. These propositions
delineate quite explicitly a static space with an existing order that is evaluated
from the partys point of view. In that sense, they are like text segments that can
be analysed and compared for time and space localisation. The following section
describes the specific model used for this purpose.
3.4.1 A discourse space model for worldview analysis
Chiltons Discourse Space Model inspired our schematic Time and Space method
of discourse analysis. The original DSM (Chilton 2004), designed for a syntactic analysis of point-of-view, shows three relative vectors for Time, Space and
Modality that emerge from a deictic centre. In our adapted model, conceptual
expressions of time and space were mapped on the Time (T) and Space (S) axes
(Figure 4). The relative distance of time and space to the deictic centre is marked
by modifiers which were projected on a Modality scale (M). For example, what
we believe to be true, right or desirable should be close and what we believe to be
false, wrong, or undesirable is distant from the ideal deictic centre. In this way, T,
S and M axes construct a relative space that foregrounds political interests, priorities and actions.
A TSM codebook was developed around the sub-corpus of introductory
paragraphs from the election manifestos of the coalition parties and the coalition
agreement. The tolerance agreement was found to be lacking explicit references
to time and space and was therefore not included. Temporal, spatial and modal
expressions in the selected texts (EM2010-Intros) were annotated manually following the codebook (Cienki et al. 2010):
1. Time was annotated for historic events, to recent developments, to the present
time, and into the future (the next government period, future generations, always,
etc.). We also included events that refer to a time-space, such as the Spanish invasion of the Netherlands in the 16th century. Where time is expressed in spatial
terms it was coded as time. The dominant deictic centre of Time was found in the
clusters Present and Future<10 (Figure 5), the time-span in which a government
has agency.
2. Space was annotated for geo-political references and its relative distance to
here, providing the scope of worldview. The dominant spatial deictic centre was
found in the clusters Local and National (Figure 6), the space in which a government has authorised agency.
3. Modality is a linguistic modifier of the relative distance between the deictic
here/now and Time and Space references that is indicative of attitude. Modality
annotation was based on a Dutch translation of Chiltons English modality scale
for deontic and epistemic expressions (Chilton 2005; Werth 1999), and complemented with expressions of desirability because they were found in the sub-corpus.
The list was then applied in a corpus-linguistic frequency analysis. The dominant
Modality clusters are characterised by empirical certitude and normative belief (to
be, to be necessary), and normative acceptability (could/might be) (Figure 7).
Time and Space references were clustered in nodes that emerged as salient
from the analysis (Figure 4). The resulting clusters were then ranked on the axes
relative to the deictic centre. The T and S clusters are relevant for EM2010 and the
coalition agreements introductory paragraphs but this may be different in other
election years.
S
Infinite
Global
Non-Western
Eternal P
Western
Past>10
EU
Past<10
NL border
National (NL)
Local
Ce
Here/now
rta
Po
A
in cce
ssi
pt
bl
Future<10
e
ab
le
Un
lik
Future>10
Eternal F
Figure 4. Adapted Discourse Space Model for EM2010 analysis,

based on Chilton (2004, 2005).
ely
M
Im
po
s
sib
le
3.4.2 Evaluation of TSM coding

The results presented here (Figures 57) concern the manifestos of the two coalition parties, the tolerating PVV, and the coalition agreement between CDA and
VVD. The purpose is to find evidence as to which of the coalition parties worldviews dominate the coalition agreement. The figures show normalised raw counts,
rather than percentages, in order to have transparency about the actual number of
occurrences in this small sub-corpus (N=8890 for the four documents). In this
study we are concerned with differences in actual usage, however, percentages
would give the same differentiation. Each figure shows one of the Time, Space and
Modality axes individually.
80
70
60
50
40
30
20
10
0
CDA
VVD
Coalition
PVV
Local
NL
NL
border
EU
WW
NWW Global Infinite
Figure 5. Relative number of Time expressions (EM2010-Intro).

16
14
12
10
CDA
VVD
Coalition
PVV
8
6
4
2
0
Very
past
Past
Near Present
Past <10
Near Future > Forever

future
10y
Figure 6. Relative number of Space expressions (EM2010-Intro).
60
50
40
CDA
VVD
Coalitie
PVV
30
20
10
0
Necessary
Desirable Acceptable Undesirable Unacceptable
Figure 7. Relative number of Modal words (EM2010-Intro) (Kaal and Maks 2011).
Results show that space is the most frequent marker of worldview: where space
has a maximum score of 72.4, Time only scores 14. All parties score higher on
the deictic centre of Government agency for all three categories: Here (NL), Now
(Present, Future <10) and Modal (certain/possible/acceptable). This confirms
their focus on the current state of affairs. According to Levinsons findings, this
type of taking perspective applies a dominant relative frame of reference with an
egocentric centre (as opposed to allocentric) and is a cultural trait of the Dutch
(and English) (Levinson 1996:114,127 ff.). His experimental findings support our
assumption that time and space are powerful rhetorical vehicles.
Some striking features emerge from these first results when we cross-reference them. The PVVs high score on Past links with its high score on deictically
close Space (Netherlands). This link sublimates a supposed traditional identity of
resilience and enterprise by juxtaposing it to current public dissatisfaction, as in
Example (3).
(3)
Steeds meer Nederlanders vragen zich anno 2010 af of hun toekomst nog wel in
Nederland ligt. [] Onze voorouders zagen de ondergelopen delta en dachten:
dit wordt een oase.
Now, in the year 2010, more and more Dutch people wonder whether The
Netherlands still holds a future for them. [] Our ancestors saw the flooded
delta and thought: this will be our oasis.
(PVV, EM2010-Intro)
The Christian Democrats and the Coalition Agreement share an interest in

Eternity: the former on religious grounds and the latter want to spare generations to come. They take a long-term future perspective for different ideological
reasons, which makes for an unusual coalition (see Eleveld, this volume). On
Space, the Coalition Agreement pays more attention to the Non-Western World
and the Global than do the coalition parties. In the current political context, this
could be interpreted as the coalition parties agreement to honour international
agreements which were backgrounded in the manifestos. On the Modality scale,
the liberal VVD articulates itself most strongly. They are optimistic and forceful,
with higher scores on Acceptable and Necessary. The Christian Democrats are
perhaps the most normative with higher scores on Desirable and Unacceptable. To
our surprise, the populist PVV articulates itself moderately in terms of Modality.
In the following section we translate the TSM results in positions onto the political
GAL/TAN dimension.
3.4.3 Translating TSM results onto GAL/TAN
The assumption is that a wide temporal and spatial reference frame includes a
broader perspective and is interpreted as being more progressive (GAL). More
space for deliberation would give room to fit more complex solutions and afford
a higher degree of tolerance to change and diversity. On the other hand, focus
on a smaller deliberation space is considered more conservative (TAN) with less
space to make solutions fit. As for Time, focus on specific national traits, based
in the far past (e.g., Example 3), is translated as Traditional. Strongly nurturing past identity indicates a Traditional-Authoritarian attitude. And prioritising
national interests is translated as Nationalistic. Combining TSM annotations can
result in different translations. As described above, Time can refer to different
positions on GAL/TAN, where both refer to tradition but not necessarily to
nationalism, depending on the spatial setting. We have noted the PVVs foregrounding of Past>10 as a period of brave founders of a nation as a Nationalistic
reference frame. In contrast, the Christian Democrats score highly on the category Past>10 on religious grounds. However, Past can also be indicative of
GAL when used as analogical evidence of patterns of change, for instance when
comparing the 1930s crisis with the current economic crisis. There is clearly an
ideological link between time and space frames and a desire for spatial proximity
to the deictic centre.
Modality results were applied interpretively to modify positioning on the
GAL/TAN scale. Past>10 with positive attribution (+) brings tradition closer to
the experience of Now; spatial EU with positive attribution (+) is closer to home
than EU with negative attribution (). In the translation schema, the value of
modal expressions has been simplified to positive (+)/ none/negative () attitude
on a five-point scale (2 to +2). Tables 4 and 5 show interpretive correlations
between Time and Space scores and GAL/TAN.
Table 4. Correlation of Time clusters with GAL/TAN.

TIME
Modality GAL
TAN
Past >10
Traditional/Nationalistic
Past and Future <10

Future >10
Libertarian
Traditional
Alternative/Libertarian
Libertarian
Eternity
Authoritarian
Green
Religious
Traditional
Table 5. Correlation of Space clusters on GAL/TAN.

SPACE
Modality GAL
TAN
Local/NL/Border
Trad./Authoritarian/Nat.
EU/WW/NWW/Global +
Infinity
Traditional/Nationalistic
Green
Religious
Applying the five-point scale, Table 6 shows a ranking of TSM on GAL/TAN with
combined TSM results in the bottom row. The CDA is moderately Traditional
and Nationalistic, the VVD is Libertarian and Nationalistic, and the PVV is
Traditional and Nationalistic, both in Time (high score on positive past) and Space
(The Netherlands). The results show that GAL and TAN are not consistent opposites, e.g., the coalition agreement is Libertarian and Nationalistic. In the following
section these results are compared to our gold standard, Vote Compass, and to the
results of the Wordscores analyses.
Table 6. TSM results and GAL/TAN positions (EM2010-Intro)
on a 5-point scale of 2 to +2.
CDA
VVD
Coalition
PVV
TIME
Modality
SPACE
Modality
TSM
Trad. (religious)
+1
Nat. + religious
+1
TN 1
Libertarian
+2
Lib./Auth. Nat.
+2
L-AN 2
Libertarian
+2
Nat.
+2
L-N 2
Trad.
+2
Auth. Nat.
+2
TAN 2
3.4.4 Cross-validating TSM with Wordscores and Vote Compass

In Section 3.2 we concluded that the words-as-meaningful-data method performed better on the Left/Right dimension than the words-as-data method. On
the GAL/TAN dimension neither method performed particularly well. In this section we compare the discourse-based words-in-context results to the Wordscores
results and Vote Compass (Table 7).
Table 7. GAL/TAN positioning of the three coalition parties.
GAL/TAN
Vote Compass
Words-in-context
Words-as-data
Words-as-meaningful data
1 2
10
VVD PVV/ CDA

VVD CDA
VVD/ PVV
CDA
PVV
VVD
CDA
PVV
Worldview analysis on TSM shows some variation with the gold standard but not
dramatically. The CDA appears less TAN, probably because their stance on Past
is based on religious grounds and is not interpreted as nationalistic in the TSM
ranking. The PVV comes out slightly more TAN on TSM because the frame of traditional Dutch resilience (Past>10) places them very traditional and nationalistic,
which reduces their libertarian stance. The most striking TSM result in this study
is that it places the PVV as more conservative than the Wordscores methods and
slightly more than Vote Compass. TSMs sensitivity to the metaphorical relationship between time and space frames and normative values provides an additional
layer of information by taking into account cognitive and narrative affordances of
discursive worldview constructions.
4. Positioning the coalition agreement
4.1
The Left/Right dimension
We now turn to the hypotheses (Box 1) and include the positions of the coalition
agreement and the tolerance agreement on the Left/Right dimension. Given the
results of the cross validation we discuss the results of the words-as-meaningfuldata analyses and leave out words-as-data results.
The words-as-meaningful-data analysis positions the coalition agreement in between CDA and VVD, which confirms Hypothesis 1: a compromise.
Ideologically it is slightly closer to the CDA, which defies the expectation that the
VVD has the largest ideological impact (Hypothesis 2). The fact that the PVV is
to the left of the CDA in this analysis might have pulled the coalition agreement to
the left, which give some leverage to the hypothesis that a considerable increase in
votes gives a party more influence (Hypothesis 3). The tolerance agreement is not
close to any of the three parties and is in fact situated between the Social Democrat
Party (PvdA) and the Christian Union (CU).
Table 8. Positioning the coalition agreement and the tolerance agreement
with words-as-meaningful data on Left/Right.
Left/Right
Words-as-meaningful-data SP
10
PvdA CU TA D66 SGP PVV VVD

PvdD
CDA CA
GL
NB: CA = coalition agreement, TA = tolerance agreement
This leaves us to conclude that on the economic Left/Right dimension, the coalition
agreement is ideologically indeed a compromise between the government parties
CDA and VVD. However, the tolerance agreement was positioned at remarkable
distance to the left of the governing parties and leads to the conclusion that the
stylistic features of the tolerance agreement made it unsuitable for the analysis and
could not be considered a like text for the Wordscores analyses either.
4.2
The GAL/TAN dimension
The coalition agreement focuses, not surprisingly, on Local and National spatial
issues but also extends to existing national commitments to the EU and international agreements (Figure 6). The high scores for time Present and Future<10
years (Figure 5) and the lack of reference to the past can be explained by the
function of the coalition agreement as a consolidation of government action for
the next term of office. And finally, on Modality, the Agreement is less outspoken,
which also makes sense, considering that the document functions to enable action,
rather than to construct attitude (Figure 7). This results in the positioning of the
coalition agreement as shown in Table 9.
Table 9. Positioning the coalition agreement with words-in-context on GAL/TAN.
GAL/TAN
Words-in-context
NB: CA = coalition agreement
CA
VVD
CDA
10
PVV
The words-in-context analysis shows that the coalition agreement is not the ideological mean of the coalition parties (Hypothesis 1), but that the largest party,
VVD, has had the most influence on the coalition agreement on the GAL/TAN
dimension (Hypothesis 2). The tolerating PVV is positioned at a considerable
distance of the coalition agreement, which indicates that, although they had the
biggest increase in number of votes in the 2010 election, they did not influence the
coalition agreement on the GAL/TAN dimension (Hypothesis 3).
5. Conclusions: A comparison of the methods
We have illustrated the advantages and constraints of layered methods of text
analysis by positioning three coalition parties and their two agreements in a political space. Testing the hypotheses required a fine-grained differentiation between
parties that had come to a not-so unusual coalition as it seems. In the process,
we have shown how the combination of a purely frequency-based method and
a lexical-semantic approach improved the positioning of political parties on the
economic Left/Right dimension against a methodologically hybrid gold standard,
Vote Compass. The extraction of deontic expressions did not, however, result in
a more precise positioning on the GAL/TAN dimension. We inferred that Left/
Right is more strongly characterised by subjective expressions of degrees of necessity, desirability and urgency of action. The positioning on the GAL/TAN dimension strongly concerns implicit beliefs and requires an analysis of the discursive
ground of party rationale. Hence, a discourse-space model for Time and Space
was developed that frames the point of view. The resulting frames provided ideologically motivated worldviews that function as rational coordinate systems for
political reasoning. This words-in-context analysis focused on time and space as
a background setting for political action and was translated into positions on the
GAL/TAN dimension.
The methods are based on three disciplines, each with their own approaches
and aims. However, the common goal was to be able to position parties with high
precision, for instance, to visualise positions accurately in voting advice applications. The two Wordscores methods need reference texts and expert surveys. A
disadvantage is that the method requires that the political positions and themes
of the reference texts must be close to those of the virgin texts because otherwise
they may miss out on unique and new political positions. This is not a requirement
for the words-in-context method, which can be applied to any text as long as it
has some form of narrative structure to indicate variations in worldview constructions. The TSM model has so far been applied as a qualitative model to test the
ground for rules that may be automated. We think it is not restricted to the GAL/
TAN dimension but could also be applied to positioning on the Left/Right dimension as a second step in the analysis, after worldviews have been identified. This
could indicate (in)consistency between worldview and issue positions. However,
rules for automated analysis are not obvious because mapping TSM results onto
political dimensions requires a clearly contextualised research frame to be able to
integrate cognitive, linguistic as well as political considerations.
The three methods have been applied at different text levels of the same corpus with the common aim to identify political positions on three levels of frames
(word frames, deontic word frames, and spatial reference frames). The politicalaction oriented lexical-semantic method (words-as-meaningful-data) extracted
policy measures at the micro (sentence) level, whereas the discourse analysis
looked into the parties perceptions of the world we exist in at the meso level
(words-in-context). Identifying meaningful text segments required linguistic
and cognitive-discourse knowledge and resulted in a more precise positioning
than taking them as a full data set in the frequency-based words-as-data method
(macro level) that disregards meaning and the rhetorical affordances of meaning
constructions. The methods are not numerically guaranteed to be unbiased but
use theories of linguistic and discursive aspects of meaning construction in texts
to approximate unbiased results.
By identifying the problem areas of each method it becomes clear that
together, the methods may give better results because they can combine qualitative
and quantitative approaches. By doing justice to lexical and textual affordances of
meaning making for stance taking, our approach seems to improve the results of
the pure words-as-data analysis. We think that adding layers of analysis to extract
how deontic words and discursive patterns create a sense of meaning and attitude
results in a more precise positioning.
References
Aarts, K. and J. Thomassen. 2008. Dutch voters and the changing party space 1989-2006. Acta
Politica 43, pp. 203234. DOI: 10.1057/ap.2008.6
Alpino Parser. Rijksuniversiteit Groningen. http://www.let.rug.nl/vannoord/alp/Alpino/
(accessed 18 June 2012)
Beigman Klebanov, B., D. Diermeier, and E. Beigman. 2008. Automatic annotation of semantic
fields for political science research. Journal of Information Technology & Politics 5, pp.95
120. DOI: 10.1080/19331680802149640
Benoit, K. and M. Laver. 2006. Party Policy in Modern Democracies. New York: Routledge.
Budge, I. 1994. A new spatial theory of party competition. British Journal of Political Science 24,
pp. 443467. DOI: 10.1017/S0007123400006955
Chilton, P. 2004. Analysing Political Discourse: Theory and Practice. New York: Routledge.
Chilton, P. 2005. Vectors, viewpoints and viewpoint shift: Toward a discourse of space theory.
Annual Review of Cognitive Linguistics 3, pp. 78116. DOI: 10.1075/arcl.3.06chi
Cienki, A. 2008. The application of conceptual metaphor theory to political discourse: Methodological questions and some possible solutions. In T. Carver and J. Pikalo (Eds), Political
Language and Metaphor. London/New York: Routledge, pp. 241256.
Cienki, A., B. Kaal and I. Maks. 2010. Mapping world view in political texts using Discourse
Space Theory: Metaphor as an analytical tool. Presented at the eighth meeting of the Association for Researching and Applying Metaphor (RaAM 8), Amsterdam. http://vu-nl.academia.
edu/BertieKaal
Entman, R.M. 1993. Framing: Toward clarification of a fractured paradigm. Journal of Communication 43, pp. 5158. DOI: 10.1111/j.1460-2466.1993.tb01304.x
Grady. J.E. 1997. Foundations of meaning: Primary metaphors and primary scenes. Unpublished PhD Dissertation, University of California at Berkeley.
Haeseryn, W., K. Romijn, G. Geerts, J. de Rooij and M. van den Toorn. 1997. Algemene Nederlandse Spraakkunst. Groningen: Martinus Nijhoff/Wolters Plantyn.
Herman, D. (ed.). 2003. Narrative Theory and the Cognitive Sciences. Stanford: CSLI.
Heywood, A. 2007. Political Ideologies (4th ed.). New York: Palgrave MacMillan.
Hooghe, L, R. Bakker, A. Brigevich, C. de Vries, E. Edwards, G. Marks, J. Rovny and
M. Steenbergen. 2010. Reliability and validity of measuring party positions: The Chapel
Hill Expert Surveys of 2002 and 2006, European Journal of Political Research 4, pp. 684703.
Kaal, B. 2012. Worldviews: Spatial ground for political reasoning in Dutch election manifestos.
CADAAD 6 (1), pp. 122.
Kaal, B. and I. Maks. 2011. Stylistics in context: Semantic and discursive aspects of subjectivity
in political texts. Paper presented at Stylistics Across Disciplines, Leiden, 16-17 June 2011.
Kriesi, H., E. Grande, R. Lachat, M. Dolezal, S. Bornschier and T. Frey. 2006 Globalization and
the transformation of the national political space: Six European countries compared. European Journal of Political Research 45, pp. 921956. DOI: 10.1111/j.1475-6765.2006.00644.x
Kriesi, H., E. Grande, R. Lachat, M. Dolezal, S. Bornschier and T. Frey. 2008. West European
Politics in the Age of Globalization. New York: Cambridge University Press. DOI: 10.1017/
CBO9780511790720
Lamond, I. 2012. Elections as points of discursive contestation: Using a critical approach to discourse as a source for empirical data for cultural policy studies. CADAAD 5(2), pp. 3953.
Laver, M. 2001. Position and salience in the policies of political actors. In Laver, M (ed.), Estimating the Policy Positions of Political Actors. New York: Routledge, pp. 6675.
Laver, M., K. Benoit and J. Garry. 2003. Extracting policy positions from political texts
Using Words as Data. American Political Science Review 97, pp. 311331. DOI: 10.1017/
S0003055403000698
Laver, M. and J. Garry. 2000. Estimating policy positions from political texts. American Journal
of Political Science 44, pp. 619634. DOI: 10.2307/2669268
Levinson, S.C. 1996. Frames of reference and Molyneuxs question: Crosslinguistic evidence.
In P. Bloom, M.A. Peterson, L. Nadel and M.F. Garrett (eds.), Language and Space.
Cambridge: Cambridge University Press, pp. 109169.
Levinson, S.C. 2003. Space in Language and Cognition: Explorations in Cognitive Diversity.
Marks, G., L. Hooghe, M. Nelson and E. Edwards. 2006. Party competition and European
integration in the East and West, Comparative Political Studies 39, pp. 155175. DOI:
10.1177/0010414005281932
Mller, W.C. and K. Strm (eds). 2000. Coalition Governments in Western Europe. Oxford:
Nuyts, J., P. Byloo and J. Diepeveen. 2010. On deontic modality, directivity and mood: The
case of Dutch mogen and moeten. Journal of Pragmatics 42, pp. 1634. DOI: 10.1016/j.
pragma.2009.05.012
Rayson P., and R. Garside. 2000. Comparing Corpora using Frequency Profiling. Proceedings of
the workshop on Comparing Corpora, held in conjunction with the 38th annual meeting of
the Association for Computational Linguistics (ACL 2000), pp. 16.
Palmer, F.R. 1986. Mood and Modality. Cambridge: Cambridge University Press.
Searle, J. and D. Vanderveken. 1985. Foundations of Illocutionary Logic. Cambridge: Cambridge
University Press.
Van Dijk, T.A. 2004. Discourse, knowledge and ideology: Reformulating old questions and
proposing some new solutions. In M. Ptz, J. Neff-van Aertselaer and T.A. van Dijk (eds.),
Communicating Ideologies: Multidisciplinary Perspectives on Language, Discourse, and Social
Practice. Frankfurt: Peter Lang, pp. 538.
Warwick, P.V. 1996. Coalition government membership in West European parliamentary democracies. British Journal of Political Science 26, pp. 471499. DOI: 10.1017/
S0007123400007572
Werth, P. 1999. Text Worlds: Representing Conceptual Space in Discourse. London: Longman.
About the authors
Michael S. Boyd teaches English Language and Linguistics at the Universit Roma
Tre and Libera Universit LUSPIO in Rome. His main research interests include
CDA, political linguistics, English for Special Purposes (ESP), multimodality and
new technologies. He is the author of various articles on political discourse, ESP
and new technologies
Alan Cienkiis associate professor of English linguistics in the Dept. of Language
and Communication at VU University Amsterdam.Employing approaches from
cognitive linguistics, his work focusses on the semantic analysis of spoken language as multimodal to varying degrees and includes applications to the discourse
of politicians and entrepreneurs. He is co-editor of Metaphor and Gesture (2008)
and Body Language Communication (2013).
Benot Collette is a PhD candidate in political science at Universit Laval and
teaches at Cgep of Rimouski. He is working on the Poltext project (www.poltext.
org) which collects and analyzes textual data sources. He has used these data with
Franois Ptry to estimate how Canadian political parties position themselves in
a political space, and how they keep their election promises.
Stefan Dahlberg is Assistant Professor and data manager at the Quality of
Government institute at the Department of Political Science, University of
Gothenborg, Sweden. His recent publications are focused on voting behavior,
democratic representation, political legitimacy and survey methodology.
Anja Eleveld has a degree in cultural anthropology, political science and labor law
from the University of Leiden. She is a lecturer and a post-doctoral researcher at the
Law Department of VU University Amsterdam where she teaches European Labour
Law and Social Law. Her research interests include the labour obligation in national
social-assistance legislations and the role of discourse in social policy change.
Laura Filardo-Llamas is a lecturer of English at the University of Valladolid,
Spain. Her main area of research is discourse analysis and conflict resolution. Her
research is applied to ethno-nationalist conflicts and to domestic violence. She
has attended several international conferences, and she has published in journals
such as Ethnopolitics, Peace and Conflict Studies, CADAAD Journal and Critical
Discourse Studies.
326 From Text to Political Positions
Steven Grijzenhout received his BSc (2009) and MSc (2010) in information science from the University of Amsterdam, both with distinction. He is currently
working as a solutions architect (EMEA, Optimizely).
Wojciech Gryc is the founder and CEO of Canopy Labs, a Toronto-based analytics
company specializing in predicting customer behaviour and trends. His previous work includes marketing, analytics, big data and business technology and
development. He has worked with McKinsey and IBM and is actively involved in
Journalists for Human Rights and Five Minutes to Midnight, providing journalism and technology training to youth groups in impoverished and post-conflict
areas around the world. Wojciech Gryc studied Mathematics and International
Development at the University of Toronto and took his MSc at Oxford University
as a Rhodes Scholar.
Graeme Hirst is Professor of computer science at the University of Toronto, working in computational linguistics and natural language processing. He was a coordinating editor of the Encyclopedia of Language and Linguistics (2nd edition) and
edits the book series Synthesis Lectures in Human Language Technologies.
Valentin Jijkoun received his MSc in mathematics and computer science in 1999
from St.-Petersburg State University (Russia) and his PhD in computer science
from the University of Amsterdam (2006). His main research topic is natural language processing, specifically, information extraction and opinion and sentiment
analysis.
Bertie Kaal is a PhD candidate at the VU University Amsterdam, The Network
Institute. Her project concentrates on a critical discourse approach to manifesto
analysis for variation in worldviews, based in cognitive affordances of spatial
frames of reference. A combination of qualitative and quantitative discourse analysis complements content and linguistic analysis in the From Text to Political
Positions project. She has a background in academic publishing and is founding
member of the Amsterdam Critical Discourse Community.
Jan Kleinnijenhuis is Professor of Communication Science at the VU University
Amsterdam, Faculty of Social Sciences. His special interests include the analysis of
news effects, economic and political news and patterns of agenda building.
Veronika Koller is Senior Lecturer at the Department of Linguistics and English
Language at Lancaster University. Her research areas are critical discourse analysis and cognitive semantics, especially metaphor, which she applies to health and
corporate communication as well as language and sexuality.
About the authors 327
Andr Krouwel is associate professor of political science at the VU University

Amsterdam. He is also Academic Director of Kieskompas (Vote Compass), where
he develops online applications that allow voters to determine which political
party is closest to their own policy preferences. His research focuses on political
parties, elections and democratisation.
Isa Maks (VU University Amsterdam, Dept of Computational Linguistics) is
developing a lexical model for a rich multi-layered subjectivity lexicon to be
used in applications within the field of sentiment analysis and (political) opinion
mining.
Maarten Marx obtained his master in political science (1990) and his PhD in
mathematical logic (1995), both at the University of Amsterdam. He (co)-authored
3 books and more than 75 scientific articles. Since 2002 his main research topic is
XML, in particular XPath dialects. His current research interest is integration of
large amounts of semi-structured, text-centric, data.
Karo Moilanen is a computational linguist, completing a DPhil in computer science at Oxford University Computing Laboratory. His research focuses on various
facets of non-factuality in text using deep, knowledge-rich linguistic modeling and
machine learning. He has been actively involved in industrial natural-language
processing as a linguist (Microsoft) and as an NLP architect in start-ups.
Nicolina Montesano Montessori is senior lecturer and researcher in the Spanish
department of the Faculty of Education at Utrecht University of Applied Sciences.
Her research interests include CDA and practice oriented research in relation to
topics such as social justice and democratic citizenship. Her work involves development of theoretical-methodological frameworks to facilitate research in the field
of social innovation and peace studies.
Franois Ptry is Professor and Director of the Department of Political Science at
Universit Laval. He is a member of the Poltext project (www.poltext.org) which
collects and analyzes textual data sources for policy analysis. He has used these
data with Benot Collette to estimate how Canadian and provincial political parties position themselves in a political space, and how they keep their election
promises.
Yaroslav Riabinin was a graduate student in computational linguistics at the
University of Toronto. Jory Graham, Magali Boizot-Roche, and Colin Morris are
undergraduate computer science students at the University of Toronto. He has
been working on high-tech patent ownership and policy making.
Magnus Sahlgren has a PhD in computational linguistics focused on scalable and

dynamic computational semantic models. Sahlgren has developed and applied
such models since 2000 at the Swedish Institute of Computer Science (SICS) and
Stockholm University. He is co-founder and Chief Scientist of Gavagai, a company
that specializes in retrieving actionable intelligence from dynamic data.
Wouter van Atteveldt is Assistant Professor in the Department of Communication
Science at the VU University Amsterdam. His interests are in political communication and (automated) content analysis. Besides working on the AmCAT content
analysis infrastructure, his current research focuses on mediatisation, especially
the tension between politicians ideals and their representation in the media.
Annemarie van Elfrinkhof is a PhD student in Political Science at the VU
University Amsterdam. Her main interest is in the dimensionality of the political
space, both in the electorate and among competing political parties. Her research
focuses on how positions can be extracted from political texts. The analysis of
policy positions of the electorate is based on a large dataset collected by the vote
advice application Kieskompas. Furthermore she is a researcher at NCDO, the
Dutch expertise and advisory centre for global citizenship and international
cooperation.
Maarten van Leeuwen is a lecturer at the Leiden University Centre for Linguistics
(LUCL). He teaches courses in stylistics, rhetoric, argumentation and discourse
analysis. In his research, he focuses on stylistic choices and their rhetorical consequences in the domain of politics, by applying insights from stylistics, discourse
studies, rhetoric and cognitive linguistics.
Piek Vossen is Professor of Computational Lexicology at the VU University
Amsterdam and Head of the Computational Lexicology & Terminology Lab
(CLTL). He is founder and president of the Global WordNet Association (GWA).
His special interests include WordNets, Terminology, Computational Lexicon
and Ontology Acquisition, Ontologies, Computational Linguistics, Big Data,
Text Mining, Language Technology and Computer-Applications. In 2013 he was
awarded the Spinoza Premium from the Netherlands Organisation for Scientific
Research.
Matthew Wall is a lecturer in politics at Swansea University. Prior to taking that
post, he worked under the supervision of Dr. Jean-Benoit Pilet as a post-doctoral
researcher on the Electoral System Change in Europe since 1945 research project
at the Universit Libre de Bruxelles. He is a Marie-Curie research fellow, having
completed a years post-doctoral research as part of the ELECDEM initial training
About the authors 329
network with Dr. Andr Krouwel. He completed his PhD thesis on African electoral and party systems at Trinity College Dublin. His publications include articles
on online politics, Irish politics and political campaigns.
Jared Wesley is adjunct professor of Political Science at the University of Alberta,
adjunct professor of Political Studies at the University of Manitoba, and Director
of Federalism and Constitution in the Government of Albertas Ministry of
International and Intergovernmental Relations. His research interests include
Canadian federalism and provincial politics, with a focus on party politics and
elections. His most recent book isCode Politics: Campaigns and Cultures on the
Canadian Prairies(UBC Press, 2011).
Index
A
abstract nouns 30, 240
agenda setting 8, 17, 201
Alpino parser 4, 306, 321
anaphora 17, 19, 55, 231, 241
associative relations 7172, 75,
8687
attitude (see also sentiment and
stance) 25, 30, 47, 118119,
132, 163, 166, 289, 295, 305
306, 310, 313, 316, 319, 321
automated methods vii, 1, 6,
28, 163, 279, 286, 293294,
298, 321, 328
B
balance theory 5, 17
blogosphere 4749, 51, 5758,
68, 7072, 7577, 8388, 91
blogs vii, ix, 4751, 5760, 63,
6669, 7172, 7677, 8688,
9394, 117118, 120, 128129,
132, 163, 251
boolean 8, 58, 98
C
Canadian Parliament 596, 99,
105, 108
Canadian parties 7, 3031, 35,
41, 114
Chapel Hill expert survey
302303, 310, 322
civil society 174175, 180, 183
coalition agreement 29, 36, 97,
189, 207213, 215218, 223,
297, 299302, 307, 312320
cognitive consistency theory
56, 17
cognitive discourse analysis
271, 310, 318, 321
cognitive linguistics 211212,

271, 272, 294
Comparative Manifesto Project
(CMP) 27, 34, 41, 155, 279
complementation 233236, 238
conceptual metaphor 16, 322
conceptual representation 12
coordinative discourse 90191,
193195, 200202, 320323
Critical Discourse Analysis
(CDA) ix, 7, 26, 137138,
163166, 169, 171172, 174,
176177, 179180, 184185,
186, 204, 211, 222, 224226,
239, 245, 247248, 266268,
271, 294, 297, 299, 325327
cross validation 18
crowd sourcing 25, 47, 53, 68,
130
D
definite and indefinite
articles31
Dfi Fouille de Textes (DEFT)
108109, 112, 114115
deictics 213, 215216, 220,
311313, 315316
deontic modality 305307, 323
discourse analysis (see Critical
Discourse Analysis) vvii,
167, 185187, 229, 250, 265,
302, 311312, 321
discourse coalition vi, 167,
189191, 193, 200203
discourse community 245, 247,
249, 326
Discourse Historical Analysis/
Approach (DHA) 166167,
169, 171172, 176, 179180,
184, 190191, 194199, 201,
203, 204, 224, 267
Discourse Space Theory/Model

167, 299, 312313, 320
discourse theory 171172, 174,
176177, 180, 184185, 187,
202203
discourse world 167, 211, 223
discursive strategies 179, 181,
207
document classification 4748,
56, 68
E
election campaign 38, 31, 85,
136, 139, 149, 277278, 283,
292, 301
election manifestos (see
manifestos)
election speech 166, 250, 253
empty signifier 167, 175176,
179, 181, 183185, 202, 209, 230
European Parliament 29, 44,
93, 107108, 110112, 115, 295
expert survey vii, 17, 3133,
3536, 39, 4144, 279280,
301303, 305, 310, 320, 322
F
frames of reference 252, 272,
322, 326
framing 4, 810, 19, 7174,
7677, 81, 8889, 168, 202,
280281, 284285, 292293,
298, 322
G
GAL/TAN 286, 302304,
308310, 312, 316320
genre vii, ixx, 3, 47, 49,
166, 239, 245251, 266, 298,
301302, 306307
Gramsci 173174, 186
H
Hansard 96, 114, 119
hegemony 137, 158, 171176,
178179, 181, 185186, 204
hermeneutics 140, 190196,
201, 203
I
ideology vii, 18, 43, 59, 67,
93107, 113115, 138, 147, 151,
155, 157, 184185, 207208,
2010212, 214223, 231,
233, 239, 248249, 255, 264,
275280, 284, 286, 292293,
295, 300301, 311, 315316,
318, 320, 323
imaginaries 171172, 185
Inaugural address/speech
46, 72, 182, 250, 253254,
263264
interpretive analysis
141, 144, 163, 165, 192, 196197,
199200
issue ownership 6, 18, 19, 73,
89
K
key concept 57, 142, 150, 176,
181, 185, 196
key words 43, 136, 142, 147, 168,
182183, 254255, 257263,
272
Kieskompas (Vote Compass)
271272, 276, 278281, 284
289, 291292, 294, 296, 299,
303304, 308310, 317318,
320, 327328
L
legitimisation 142, 171174, 181,
207208, 211214, 216218,
220, 223
lexical-semantic approach
115, 164165, 297, 305, 320321
lexicon 66, 99, 123, 126127,
129130, 132, 228, 327328
life-course perspective
189190, 193194, 196198,
200, 202
linguistic checklist 168, 225
229, 233, 237238, 240
Linguistic Inquiry and Word

Count (LIWC) 106, 115
M
Machiavelli viii, 2, 4
machine learning ix, 748,
5355, 65, 68, 117, 123124,
126130, 133, 327
manifestos vii, ix, 3, 6, 2728,
3032, 3544, 98, 135139,
155, 158, 271, 275276, 279,
282283, 287, 289, 297307,
310312, 314, 316, 322, 326
manual coding 11, 17, 56, 120,
135, 152, 163, 166167, 254, 293,
303, 312
McCain 8, 57, 105, 115, 267
Mechanical Turk 5154, 69, 130
metaphor 167169, 204, 207,
214, 216220, 222223, 231,
239, 242, 249250, 272273,
311312, 318, 322, 325326
multiple audiences 167, 207, 222
myth 175176, 179, 181, 183184
N
narrative ixx, 138140, 150,
157, 164, 169, 171173, 175176,
179, 181, 183185, 187, 190193,
197198, 200201, 310311,
318, 320, 322
Natural Language Processing
(NLP) 47, 49, 5051, 6970,
76, 115, 132133, 326
Net-method viii, ix, 11, 26
network theory 1, 70
new media 165, 245247, 249,
251252, 265266
nodal point 75, 183184
O
Obama 78, 1112, 4749, 52,
5657, 6364, 66, 68, 94, 105,
115, 167, 245246, 249, 251
254, 256258, 260, 262266
ontological 140, 179
ontology 1, 68, 1011, 1415,
17, 140, 174, 179, 185, 328
opinion mining 54, 69, 117
119, 132133, 327
outsiders 7174, 7689, 94
P
parallel textual data 2729,
31, 43
parliament 9, 44, 78, 82, 93,
95108, 110112, 114115, 119,
130131, 226227, 267, 273,
285, 295, 298, 300301
parliamentary debates 2425,
115, 279, 298
parliamentary proceedings
ix, 117121, 131, 132
parliamentary speeches
vii, ix, 96, 101, 166, 227
party manifestos (see manifestos)
party websites 76, 87, 279, 283
point of view 235, 311312, 320
political dimensions
viiviii, xi, 24, 27, 31, 3335,
38, 41, 273, 279, 280, 282,
284, 286287, 291292, 295,
299300, 302305, 307310,
316, 318321
political speech 100, 114115,
245, 247, 249251
politics of attention 16, 18
Poltext 135136, 149, 155, 325,
327
positivism 141142, 144
post-positivist analysis 191192
predication 46, 1112, 1416,
167, 194, 199, 240
Progressive 32, 95, 97, 241,
272, 284, 286287, 291292,
299300, 302303, 310, 316
proximity theory/values
20, 84, 8687, 213, 289,
291292, 316
proximity approach 289, 292
Q
Qualitative Document Analysis
(QDA) ix, 135136, 138,
143148, 152156
R
random indexing ix, 25, 59,
7172, 75, 8384, 8889
re-imagining 49, 265
referential expressions
212, 232233, 238
Index
333
rhetoric ix, 19, 2324, 4950,

71, 136140, 149152, 155156,
158159, 164, 169, 187, 204,
223, 225226, 232, 234, 236,
238239, 241, 267, 311, 315, 321
Right-wing 54, 79, 81, 108110,
113, 193, 226, 272, 305
S
semantic network 11, 14, 1718,
20, 55
semantic relations 5, 20, 55,
7072, 75, 83
semantic web 5, 18
sentiment analysis ix, 14, 17,
4749, 5255, 60, 63, 6570,
115, 117120, 122, 127129, 133,
168, 326327
social cognition 69, 208, 211
social imaginary 175176
social network modeling/analysis
25, 4748, 51, 5760, 67, 68
socio-cognitive approach
165169
spatial mapping 209, 212,
272273, 291292, 295, 299,
310318, 320328
stance vi, vii, 44, 47, 163,
165166, 234, 273, 277278,
279, 281, 283, 288, 289, 291,
294, 298299, 305, 306, 311,
318, 321
stemming 3132, 37, 4041, 97
storyline (see narrative)

164, 167, 189190, 192193,
195197, 199202
stylistic theory/analysis
ix, 141, 164165, 225229, 233,
237238, 240, 319, 322
subjectivity ix, 49, 117121,
123129, 132133, 141, 143145,
186, 297, 299, 312, 322, 327
Support Vector Machine (SVM)
24, 60, 9394, 96, 124, 128
T
temporality 38, 49, 55, 164,
209, 217
text mining 8, 65, 108, 328
text-world theory 165, 168, 171,
180, 184187, 207, 211224, 323
text as data 34, 144, 271, 297
time and space 217, 310313,
315316, 318, 320
transitivity principle 13, 17, 115,
167, 213, 220
triangulation 146147, 154, 180
triplets 6, 14
trustworthiness 144146, 153
V
vector model (see support vector
machine)6
victory speech 246, 250,
253255, 257, 264
Vogelaar 226237, 239, 242
Vote Advice Application
271, 295, 296
Vote Compass (see Kieskompas)
W
Weka toolkit 60, 67, 69, 124,
127129, 133
Wilders 225237, 239, 242
word count 18, 38, 82, 106, 115
word frequency method
297298
word space 7576, 8389, 212
Wordfish ix, 2729, 3132,
3842, 45, 279, 286, 298
words as data 43, 299, 304,
308310, 321322
words as meaningful data
299, 305
words in context 310, 318321
wordscores ix, 2729, 3132,
3439, 4143, 45, 127, 279,
286, 298299, 302305,
307310, 312, 317320
worldview 173, 185, 272, 279,
299, 303, 310313, 315, 318,
320321
Y
YouTube 167, 245247, 249
253, 257, 261, 265267
Z
Zapatista movement (EZLN)
171176, 178181, 183184

(Discourse Approaches to Politics, Society and Culture 55) Bertie Kaal (ed.), Isa Maks (ed.), Annemarie van Elfrinkhof (ed.)-From Text to Political Positions_ Text Analysis across Disciplines-John Ben (1).pdf

Enviado por

Dados do documento

Descrição original:

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

(Discourse Approaches to Politics, Society and Culture 55) Bertie Kaal (ed.), Isa Maks (ed.), Annemarie van Elfrinkhof (ed.)-From Text to Political Positions_ Text Analysis across Disciplines-John Ben (1).pdf

Enviado por

Direitos autorais:

Formatos disponíveis

From Text to Political Positions