Surveys

1
Chapter 10

Quantitative research form of planned inquiry

Research design: -engage in an intensive process of formulating the question
-operationalizing the measures
-arrange the evidence gathering
-analyze procedures
-combines creative problem solving abilities with analytic rigor
and insight
-important to identify the key abilities undelaying the
investigation and the ongoing explanations of evidence generated by
other researchers
-becomes a formative moment of theory building

Quantitative research pay attention to three tasks:
1. Operationalization (ensure that they make the right observation): the process
of defining research questions, developing measurements instruments and
gathering empirical evidence in a data set
2. Generalization (a sampling strategy, so that they can generalize from the
observed universe to the population): the use of basic sampling and statistical
techniques to estimate the likelihood of occurrence of a phenomenon in the
general population, it involves developing a sampling frame, setting the sample
size and estimating the sampling error in the data set.
3. Inferential analysis ( concerns the application of inferential statistics, to
manage the uncertainties that researchers face in explaining any phenomenon):
embodies a series of evaluative arguments concerning the relationship between
the independent and dependent variables based on rigorous ways of comparing
and testing data against hypothetical models of possible outcomes; inferential
analysis uses the statistical tools of frequency distribution, measures of central
tendency, cross tabulation and Chi-square to examine the patterned relationship
between separate variables.

Example
RQ: Does TV viewing distort peoples estimates of affluence in the real world?
Theoretical issue identified: Why do we think a relationship exists between TV
viewing and estimates of affluence?

STEP 1
Four types of research questions:
1. Reportage: What is going on here?
2. Explanation: Why is it happening?
3. Description: What does it mean to those involved?
4. Evaluation: Is it on balance a good or a bad thing?
When explanation and evaluation are at stake, researchers often refine their
questions into a series of hypotheses.
Hypothesis: a carefully constructed and highly specific articulation of the key
expectations of the research question.
2 Surveys

2
Null hypothesis: the question stated in the form that the expected relationship
does not exist.
Alternate hypothesis: the question is stated that the relationship does exist.



STEP 2 & 3

The central methodological issue in operationalization is construct validity.
Construct validity refers to the degree to which measurments actuayll measure
what we say they do.
Impotant distinction between: beliefs, attitudes, intentions and behavior.

Designing a question is a matter of thinking about how to communicate with
respondents. The researcher/interviewr encodes meaning in the form of a
question and the respondents interpret or decode the intent and the meaning of
the question.
Use a dictionary in order to measure each term.
Ask friends of what they think defines a word.
This what their answers have in common.

Quantitative researchers must translate verbal responses into numbers, for the
purpose of analysis.

STEP 4: Specifying the independent and dependent variables

In deciding whether one variable depends on another, the researcher must think
descriptevly or causally (not stating that one variable completely determines the
other, only that it influences it).

STEP 5: Choosing the level of measurement
Distinguish between:

Nominal variables:
- based on categorical distinctions
-most frequently encountered form of measurement in social sciences
- asks: In which category would you fit yourself?
- can be used to provide a summary account of the distributions across
populations (percentage)

Scaled variables:
- based on the intensity or magnitude of an atribuite
- need a numeric system to denotation
- ask: Where along this continuum your perceptions fall?
- can be constructed in ordinal, interval and ratio scales

Ordinal variables:
- any idea that is operationalized as an ordered set of terms
- respondents have the ability to sort & rank a variable
Surveys
3
- must use rudimentary descriptive statistics in assessing data

Interval scales:
- imply a dimensional order of magnitude, but assume equal intensities
between the value 1 compared to 2 as between 3 and 4.
- True zero does not exist
- Two types of interval scales: with a midpoint & without one.
- Midpontins are favoured if you want accurate readings (people place
themselves in positive / negative orientation when answering questions).

Ratio scales:
-imply equal intervals between stated degrees on the scale, but have an absolute
zero.
-assume that it is meaningful that someone has nothing of the trait being
measured.

Index / data construct: when two or more indicators are linked.

STEP 6: Designing the sample
-decide on what you are going to communicate
-obseriving around the clock = best valid observations
-think carefully about both social and temporal dimensions ins ampling
procedures

Sampling:
-taking a subset of a total population that is enough like it that valid
generalization can be made to the total population.
-how you select the subset will determine: how you may compare, contrast and
generalize about how widespread your findings are in the general population.
- make sure that who we observed is not biased by the cristeria selection (to do
that select randomly samples)

Framing samples:
-definying the target population or universe for the sample (think about the
basic demongraphic ot behavioral characteristics that may affect the RQ)
-locate the sample frame ((a complete list of the total population: voters lists,
directories etc.)
-choose a random probability or non-random sampling method (probability vs.
non.probability samples)
Probability samples involves selecting individuals in such a way that their chances
of selection are known.
A probability is a projection or estimation expressed as the number of chances or as
a percentage of times the outcome will be observed by chance if randomly sampled.
Stratified randomly sampling (different socio.economic levels are adecvately
represented).
Multi-stage stratified sampling (based on each nations census)
-decide on the sample size


4 Surveys

4
STEP 7: Mapping and analyzing the data
-use probability theory& statistics
-look for patterns
-examine relations between variables
-test the hypotheses
-for large data sets use computer programs

MEASURES OF CENTRAL TENDENCY
-is based on the probability that responses will be randomly scattered around a
center cluster, evident when all date is graphed.
-describe results as: a frequency chart, indentify which answer category occurs
most often, the middles case relevant in analyzing orinal or ratio scales

MEASURES OF DISPERSION
Standard deviation= a concept to describe how cases are dispersed, enabling us
to compare similarities and differences in distribution for scalar variables across
groups
High numbers =values are higly dispersed
Low numbers = they are grouped closely around the mean

ANALYSIS DATAT RELATIONSHOPS: THE CROSS TABULATION OF NOMINAL
VARIABLES
After mapping responses to variables, we can start exploring how variable
interact by checking if the sampling of gender was similar to that of the known
general population.
Use CHI Square (used to test relationship between 2 variables). Based on a
null hypothesis. It is the most versatile statistic for both nominal and ordinal
scales.

EXPLORING ASSOCIATIONS OF VARIABLES
Suggesting that there may be stronger cultural variation in use by age and
genders.
Correlation coeffiecint: the statistic we use for testing any linear equation
between 2 variables.


STEP 9: Evaluating causal raletionships
STEP 10: Reporting findings
Empirical quantitative research requires three arguments:
-the validity and reliability of our evidence
-meaning or interpretation of the evidence
-relationship between our concepts

Research report steps:
Introduction:
-state the framing of the RQ
-justifies why it is important to investigate it
-explain how the concepts are defined and operationalized

Surveys
5
Method:
- explain the rationale for the design of the study
- justify the research method
- define the sample frame (note how subjects were selected and
approached)
Results:
-report on the data findings (provide charts/ tables that map the data)
-set out analytic procedures that were used to address particular arguments
(research beliefs and findings)
-summarize results
-explain conclusions

Discussions / Findings:
-explore the implications of the main findings
-explain how findings relate to the original RQ
-comment on the validity / reliability

Chapter 11

Surveys allow us to quantify and interpret rather abstract patterns of culture,
serving as a map against which the dynamic character of an individual taste
culture or lifestyle may be brought alive.

Surveys represent the drive in the human sciences to categorize or generalize
social phenomena, the will to find unity out of the buzzing confusion and
diversity of everyday life.

Surveys may aspire to deep description, offering a strong tool in critical audience
research.

Methodologically, the challenge facing surveys is to use them in cross-cultural
comparison, and to further perfect the operationalization of key measures to do
with class, cultural capital and cultural mobility over time.


Chapter 12

Surveys:
-gathers information from people by asking them questions about themselves
-surveying implies space, viewing a whole, such as a social landscape, from a
particular vantage point
-scale and scope theoretical / methodological metaphors to describe the place
survey work occupies in the terrain audience studies
-rooted in geography, describing characteristics of people in some special
location

Scale:
-refer to territorial span/size
-implies sampling / rules for generalizing
6 Surveys

6
Scope:
-measures breadth

Variables:
- possible patterns of convergence or divergence in audience practices


Advantages
-flexibility
-conducted in natural settings
collect large amounts of data over large samples
-standardize measurement protocols to enable comparison across very different
cross populations
-cost effective
-high in reliability
-ease of replicability
-advantage of debate over scientific discourse


Disadvantages
-independent variables can not be manipulated in natural settings
-weak in validity
-flattening qualitative ambiguity / interpretation of the context meaning
-big gap between what people say vs. what they do
-not useful in understanding behavior / observational/experimental tecniques
-not a good way to examine audience changes over time
-survey work in the academic realm in expensive

Surveys became central to the workings of culture and power in the study of
audiences.
Privatization, individualization and globalization have escalated with important
consequences for how we understand social identity, cultural taste and cultural
influences on the audience.

Privatization:
-the changing political economy of entertainment, where commodified,
deregulated entertainment forms are growing
-the changing practices of consumption, where consumers are increasing
entertainment spending and directing it to new media
-the deeping ideology promoting consumption, in which audiences are conceived
of primarily as buyers or target markets

Individualization:
-the decontextualization of the social experience of TV viewing

Globalization:
-the flow of peoplescapes, with increasing immigration, as it does
screenscapes, with growing imports and exports of TV products and services.

Surveys
7
Census: a way of finding out about a group of people and collect information
about them.
The most important use of a census is in mapping the major characteristics of a
people: age, gender, race, family, structure, working status, patterns of
emigration or immigration, life cycle, education, income.

State produced audience statistics:
- sampling frame (use of census)
- social access to media technology (how many citizens have access to
media technology int heir home)
- time (activities over a life cycle)
- consumer patterns of spending (income directed towards media &
culture)
- cultural participation

Online entertainment market analysis is the premium form of market
intelligence in audience research today.

Key Concepts in commercial audience measurement:
-penetration (percentage of population with the technology to access a medium)
-daypart (no. of people watching TV)
-reach (special dimension of audience studies)
-reach and frequency analysis (net size of audience and number of times a
person is exposed to a particular advertising)
-rating (percentage of the total audience with access to the medium who have
seen / heard the specific message at a specific time of day)
-rating point (where one percentage point is translated to actual thousands of
people reached
-share (percentage of the total audience who have their sets on / actually using a
given medium and have seen or heard a specific message
-cost per thousand (basic unit of comparison; the cost of reaching 1000 viewers
or listeners with an advertising message

The future of electronic audience measurement is passive meters. Passive
surveillance technology involves some kind of infrared sensor or surveillance
transmitter or receiver to distinguish individual viewing, trap location and adapt
to capturing simultaneous multi-set use.


Chapter 13

1. Designing surveys as a snapshot in time

Surveys ask people questions and accept their answers as data. They look for
general patterns rather than exploring in depth why a particular person did a
particular thing.
Survey design must be appropriate to your RQ.
Surveys are used if your umbrella question is:
- implicitly representative (how many people would watch?)
8 Surveys

8
- describes a relationship (women use internet less often than men)
- -seeks to influence something
Be sure to understand your surveys pupose:
-to discover
-describe
-map
-explain

Start by defining a general RQ:
-why is it an important problem
-what you mean by the terms you use
-what other factors or subsidiary RQ may be connected

Find focus by asking the following questions:
-Is this topic researchable, given the time, resources and availability of data?
-Do I have a personal interest in the topic to sustain my attention?
-Will the results of the study be of interest and use to others?
-Does the study fill a void, replicate, extend, to develop new ideas in the scholarly
literature?
-Is the topic likely to be publishable in a scholarly journal?

2. The Survey instrument: choosing the lens

Asking survey questions:
-directly
-indirectly

Direct interviewer-administered formats:
-face-to-face
-personal
-telephone
-computer assisted telephone interviewing (CATI)

Face-to-face interviewing is useful when:
-the ideas explored are complex
-the interviewer presents visual prompts to enhance answer quality
-interpersonal interaction can motivate respondents and improve response rates
-the technique is slow & expensive
-carries personal risk if the interviewer travels to neighborhoods he may not
want to
-low cooperation rate
-some may unconsciously intimidate and get truncated answers
-some may administer the questions in subtly different ways so different
answers are obtained
-age, gender and race can introduce subtle cultural bias in the joint construction
of meaning
-interviewer induced variance is thus a large part of survey sample error


Surveys
9
Telephone interviews:
-used when studies are time sensitive / opinion need to be mapped repeatedly
over time
-advantage of being relatively inexpensive /fast/amendable to random sampling
-can be undermined by answering machines
-competition from telemarketing and the need to keep it short and simple
-basic telephone survey interviewer works with pen / paper
-computer assisted telephone interviewing (CATI) interviewer works from a
monitor and software guides administration of the survey
-CATI is used for larger sample sizes since it is easier to standardize
-response rate rapidly declining

Indirectly administered formats:
-include: mail, internet surveys
-ask written question and rely on the respondent to fill them out
-self completed questionaries are often better for sensitive areas where answers
may be embarrassing / where extensive information is required
-often used in educational / health research where personal anonymity
encourages disclosure
-low in response rate
-questions often answered out of order / not answered at all
-less likely to elicit responses to more open ended question (the more work a
respondent has to do by himself, the lower the response rate).

3. Ethics in securing cooperation with data subjects
Winning agreement to participate involves personal persuasion and establishing
trust for the studys objectives and the interviewer.
You need to establish your credibility, and declare who you represent.
-Why did you select them?
-What is it you want to talk about?
-How much time will it take?
They also have a right to know what you will do with the personal information
collected.
Since survey research has become deeply contaminated by commercialization-
and abuse of personal, confidential information- there is a greater onus on the
researcher to legitimate the inquiry.

The introduction to your survey should reveal:
-introduction of interviewer and institution
-how the respondent was selected
-topic and purpose of study
-frame of respondent protection (voluntary cooperation / answers kept
confidential)
-likely length of interview

Too much disclosure can bias the results.
Psychological processes are modifiable in people know they are being studied.
Special ethical considerations arise when there is risk of harm from disclosure of
potentially criminal behavior, risk of psychological discomfort arising from
10 Surveys

10
participation in research about a sensitive topic, or research conducted among
minors.

Concern for respondents:
-How will confidentiality/anonymity be assured?
-Who will have access to the results of the study?
-Is the subject matter likely to be painful in any way for the respondent?
-Is there any possibility that an informant will think I have the power to help or
harm him or her?
-Is the questionnaire intrusive? Is so, by what right do I intrude?
-Is the research consensual? Whose consent is being sought? Are the
implications of consent fully understood?
-Is the outcome of the research likely to be harmful to respondants?
-Is there a conflict of research interests in this situation, and whose side am I on?
-Does the research reinforce a dominant power relationship or stereotype to the
detriment of respondents or other parties?
-How do you plan to destroy the data?

Ethical issues:
-confidentiality
-removal of identifiers
-anonymity
-disclosing information about the purpose / sponsor of the survey
-proper interviewer training
-ensuring informed consent
-traumatic topic- offer post interview support

Ethical considerations embedded in questionnaire design:
-indirect questions prove more effective
-lessening imputation of deviance
-lessening psychological immediacy via. Substitutions of reporting on others
-avoiding emotionally loaded or taboo words

4.Epistemological basis for the survey gaze
Use of verbal data, qualitative or quantitative, has been an important foundation
for audience research.
The general epistemological problems with collecting verbal data are:
-halo effect
-non-attitude
-faulty recall

Halo effect
An interview on culture and the media can be seen as a symbolic battlefield for
cultural capital and social status. We know that interviewers, especially
university researchers, carry social status in interaction with their subjects,
which may shape peoples desire to cooperate, willingness to please, ability to
disclose and type of answer given.
Four way to manage the halo effect:
1. Pre-testing language
Surveys
11
Asking respondents to think aloud when answering or to restate the
question in their own words.
2. Obvious design flaws
Magnify the halo effect.
3. Legitimate guilty pleasure disclosure
Making non-conformity more acceptable in the body of the question. To indicate
that some people like romance novels, for example, while others do not, begins
to legitimate low cultural tastes.
4.Measure things directly
When trying to measure traits of personality that may relate to media use, we
seldom ask, Do you suffer from low/enjoy high self esteem, in exploring the
views on ads in fashion magazines, instead ask a variety of indirect questions.

Faulty Recall
There are cognitive barriers to be overcome in the design of audience surveys;
respondents may not recall the specifics of a media interaction, or may answer
when they really do not know.
Administer visual prompts (magazine covers).
Ask if people have seen a particular ad-recall period must be short.
Questions are often posed in terms of visualizing the daily routine for media
consumed, while getting up, preparing for work, travelling to work, or,
conversely, epochal personal and family occasions.

Low Salience
The most serious problem is when respondents are not sufficiently interested to
recall something, or you are seeming to force them to talk about something they
have chosen to ignore.
1. Allow the respondents to skip sections of the questionnaire they dont
care about or have no interest in.
2. Understand cues for salience and how to probe for interest from the
respondent.

Managing the non-attitude
Interpreting answers such as dont know, cant recall, undecided, no opinion,
it depends and neutral is all important. These represent quite different
knowledge states.
You do not want your respondents to express an opinion, even when they do not
have one, inspired either by a mistake, blindly choosing and answer at random,
imputing meaning from the context of the questionnaire or answering by
analogy to other firmly held views.
Education and involvement have proved the best predicators to non-substantive
responses. Those with less formal education are more likely to confess not to
know. Those with little interest in an issue are also likely to be predisposed to an
easy opt-out.
To manage the non-substantive response, there are two opposing techniques:
asking if people have thought about an issue before posing and attitude question,
or asking how strongly they feel about something after they express an opinion,
to isolate centrality of views.

12 Surveys

12

Structure and standardization in encoding meaning: setting the F-stop
The level of structure or standadization increases with the scale of the
interactions. In surveys there is a continual trade-off between open, reflexive and
narrative talk and the need to standardize responses for analysis.
Standardized survey questionnaires cost less, in term of both time and money,
than personal in-depth interviews. They easily allow data comparison across
widely varying populations, and delivers better potential for generalization.

Surveys present a continuum of structuring questions, ranging from open to
closed.

Open-ended question:
-seeks and answer the respondent is free to construct
-much more difficult for respondents to answer
-often avoided in questionnaires
-the answers may measure in thousands of pages
-respondents have to think harder to choose and frame their answers
-analysts have to work harder to sort and name the groups of answers
-help introduce variation in questionnaires
-only way to explore language issues at a broad scale
-used to develop close questions or indexes
-rarely subjected to statistical analysis in survey research (cannot be
meaningfully compared across populations)

Closed ended question:
-fix the categories of choice for the respondent
-analytic costs
-they elicit answers where no opinion or knowledge exists
-they may over-simplify or distort issues
-they may force or coerce an answer
-they may be boring to answer
-easier to administer /decode/process for large samples

Considerations of questionnaire design:
-ethical sensitivity
-validity (making sure the measure you select is measuring the concept you need
to measure)
-reliability (when responses are consistent over time)

Finding your demographic (first steps)
1. Demographic information (gender, age, household size, racial
information, employment status, income, occupation)
2. Behavior (what people say, do , use or buy)
3. Cognitive (referring to how people think)
4. Affective (information that is expressive of emotion)


Design guidelines:
Surveys
13
1. Ensure your questionnaire reads well
2. Use simple language
3. Be specific
4. Provide interviewer instructions
5. Provide transitions for changes
6. Make sure the order moves from general to particular
7. Avoid and, or double-barreled questions with any conjunctions and in
them
8. Avoid emotionally and ideologically laden terms
9. Provide counter arguments
10. Use multiple measures of the key analytic variables and use internal
controls

Non-probability methods:
Purposive:
-Most frequently used
-study leaders of the internet
-identify them through observation
-invite them to participate in a survey
Snowball:
-network / referring
Convenience:
-people are being interviewed at a hip caf downtown
Self selection:
-those who reply to a bulletin board posting about a study