Questionnaires and Structured Interviews

Chapter 9

Introduction
Questionnaires and structured interviews

are the two most widely used methods of data collection in social science research.

Introduction
Questionnaire

A data collection instrument with questions and statements that are designed to solicit information from respondents

Introduction
Structured interview

A data collection method in which an interviewer reads a standardized list of questions to the respondent and records the respondents answers

Introduction
Survey

A study in which the same data are collected from all members of the sample

Usually in the form of answers to questions Often use large probability samples and crosssectional study design

Introduction
Respondents

The participant in a survey who completes a questionnaire or interview

Introduction
Self-report method Another name for questionnaires and interviews because respondents are most often asked to report their own characteristics, behaviors, and attitudes.

The Uses of Questionnaires and Interviews

Questions can be asked about a variety of topics

It is common to use the self-report method to gather

information on attitudes, beliefs, values, goals, and expectations Questions can also be used to gather a persons level of knowledge on something or behavior

Introduction
Stop and Think

The surveys by Public Agenda and Pew focused on attitudes towards national energy policies and health care, respectively.

Suggest a list of other topics that you think you could ask questions about using a questionnaire. Would you be concerned about the accuracy of the answers on any of these topics?

Focal Research
Environmentalism Among College Students:

A Sociological Investigation by Brandon Lang and Christopher W. Podeschi

Asked college students questions about environmental issues Interested in whether the students attitudes were connected to some background characteristics and to behavior patterns Descriptive and explanatory purposes

Focal Research
Ethics

The researchers submitted a proposal to their universitys Institutional Review Board (IRB) where it was approved. The respondents were informed about the studys sponsor, were assured that the surveys were anonymous and that they could skip any of the questions.

The Uses of Questionnaires and Interviews

The Uses of Questionnaires and Interviews

Although surveys are widely used there are

important concerns about their validity When using self-report methods it is based on the implicit assumption that people have the information being asked and they will answer based on their core beliefs and values

The Uses of Questionnaires and Interviews

Answers to questions may be compromised if the

participants

Do not understand Do not take the time to adequately answer the question Influenced by current mood, events, etc.

For example, research has shown that people who are in more positive moods answer more favorably compared to those who are feeling more negative (Martin, Abend, Sedikes, & Green, 1997)

The Uses of Questionnaires and Interviews

Measurement error

Error that occurs when the measurement we obtain is not an accurate portrayal of what we tried to measure The giving of inaccurate answers to the questions

The Uses of Questionnaires and Interviews

Measurement error may also be affected by

question wording respondents level of knowledge on the topic whether or not the respondent has an opinion about the topic respondent having an accurate memory respondent wanting to present a good image

Participant Involvement
Response rate

The percentage of the sample contacted that actually participates in a study

Participant Involvement
Response rate is affected by

The number of people who cannot be reached for the study (noncontacts) The number who choose not to participate (refusals) The number who are incapable of performing the tasks required of them

For example, due to illness or language barriers

Participant Involvement
Nonresponse error Results from differences between nonresponders and responders to a survey

Participant Involvement
When participants and nonparticipants differ

in social characteristics, opinions, attitudes, values, or behavior then generalization to a larger population is much more difficult Some researchers have questioned whether higher rates of refusal automatically means more bias

Participant Involvement
Stop and Think

Have you ever been asked to complete a questionnaire or an interview? What factors did you consider when deciding whether or not to participate? Think about what you would do if you were on the other side. What would you do to try and convince someone to participate in a survey?

Participant Involvement
Participation in studies can best be

understood within a social exchange context.

Social exchange means a cost/benefit approach Once potential sample members are contacted, they must decide about cooperation after thinking about the costs and benefits.

Participant Involvement
Respondents are asked to give up their time, engage

in interactions controlled by the interviewer, think about issues or topics that might cause discomfort, and take the risk of being asked to reveal embarrassing information. Potential participants might worry about privacy and lack of control over the information

Participant Involvement
Conversely, participants might want to

participate because of an interest in a topic, a desire to share their views, or knowledge that their information will be useful to science and society

Participant involvement
Possible ways to increase participant rates

Advance mailings Incentives Giving small gifts Token cash payments Focusing on the interesting aspects of participation Re-contacting participants to encourage participation Minimize the costs of participation For example, time and possible embarrassment

Self-Administered Questionnaires
Self-administered questionnaires A questionnaire that the respondent completes by him or herself Can be written or web-based survey

Self-Administered Questionnaires
Interview A data collection method in which respondents answer questions asked by an interviewer Can be done by phone or in person

Self-Administered Questionnaires
Group-administered questionnaires Questions administered to respondents in a group setting

Self-Administered Questionnaires
Group-administered questionnaires

Advantages Allows the researcher to explain the instructions and answer questions. Provides the researcher some control over the setting in which the questionnaire is completed Allows the respondents to participate anonymously
Helps in getting honest answers to sensitive questions

Usually results in a good response rate Inexpensive

Self-Administered Questionnaires
Group-administered questionnaires

Disadvantages

There might be no group setting for the population the researcher wants to study A group setting might raise concerns over the violation of voluntary participation Groups typically have limits for the amount of time they spend on a survey

Self-Administered Questionnaires
Mailed questionnaires Questionnaires mailed to the respondents residence or workplace

Self-Administered Questionnaires
Mailed questionnaires

Advantages Inexpensive Reasonably effective No time requirement, allows the participant to not feel rushed Answering in private, creates fewer social pressures and expectations

Self-Administered Questionnaires
A questionnaire can be individually administered or

administered as an Internet questionnaire An individually administered questionnaire is hand delivered to the respondent and picked up after completion Similar to using mailed questions, but the expense is higher and has a better response rate

Self-Administered Questionnaires
An internet questionnaire is sent by e-mail

or posted on a website

Self-Administered Questionnaires
Coverage error

A sampling error that arises when the sampling frame is different from the intended population

Self-Administered Questionnaires
Cover letter

The letter accompanying a questionnaire that explains the research and invites participation

Interviews
Structured interview

Interviewer reads a standardized set of questions and the response option for closed-ended questions.

The interview has some similarities to a

conversation, except that the interviewer controls the topic, asks the questions, and does not share experiences or opinions.

Interviews
The set of instructions to the interviewer, the

list of questions, and the answer categories make up the interview schedule The use of structured interviews allows for some flexibility in administration, clarification of questions, and the use of follow-up questions.

Interviews
Interviews typically have good response rates

Interviews are more expensive than questionnaires,

because interviewers have to be hired and trained. Using an interviewer adds the potential of interviewer effect, or the changes in respondents behaviors or answers that result from some aspect of the interview situation

Interviews
In-person interviews The dominant mode of data collection from 1940 to 1970. An in-person interview is conducted face to face. In-person interviews are a good choice for questions involving complex reports of behavior, for groups difficult to reach by phone, or for respondents who need to see material or to consult records as part of the data collection, or when the interview is long, and high response rates are essential.

Interviews
In-person interviews

Rapport

A sense of interpersonal harmony, connection, or compatibility between an interviewer and a respondent

Interviews
Phone interviews The dominant mode of survey data collection since the 1970s Preferred because it can yield close to the same results as an in-person interview at half the cost Good for people who feel too busy for face-to-face interviews New technology allows makes it cost effective to do computer-assisted telephone interviews in which data are collected, stored, and transmitted during the interview

Constructing Questions
Researchers make a series of decisions

based on assumptions about how respondents read or hear the questions that are asked.

Constructing Questions
Types of questions

Open-ended questions

Questions that allows respondents to answer in their own words

Constructing Questions
Types of questions

Closed-ended questions

Questions that include a list of predetermined answers

Answer categories must be exhaustive and mutually exclusive

Constructing Questions
Types of Questions Deciding whether to use open- or closedended questions involves several issues.

Answer choices can provide a context for the question, they can make the completion and coding of questionnaires and interviews easier. Respondent might not find the response that best fits what they want to say, and answer categories can be interpreted differently by different respondents.

Constructing Questions
Types of questions

Open-ended questions

All open-ended responses must be categorized before the researcher does statistical analysis A limited number of answer categories must first be created for each question, so the data may be coded

Constructing Questions
Types of questions

Coding

The process of assigning data to categories

Coding is a time consuming and expensive task that can result in the loss of data richness.

Constructing Questions
Types of questions

Screening questions

Question that asks for information before asking the question of interest Example
Have you applied to a graduate or professional program? Yes or No If yes, please list the program(s)

Example
It is important to ask someone if they voted in November, before asking whom they voted for.

Constructing Questions
Types of questions

A screening question is often followed with a contingency question A contingency question

A question that depends on the answer to the previous question.

Example If yes, go to question #___ Example Do you work for pay outside the home? If yes, how many hours per week are you employed?

Constructing Questions
Types of questions Occasionally researchers use indirect questions, where the link between the information the information desired and the question is not obvious This is done by creating scenarios or vignettes about people or situations that the researcher creates to use as part of the data collection method.

Constructing Questions
How to ask questions

Keep the questionnaire or interview as short as possible Only the necessary questions planned for data analysis should be asked

A preliminary draft or pilot test should be used

with a small sample

A pilot test is a preliminary draft of a set of questions that is tested before the actual data collection.

Constructing Questions
How to ask questions

Avoid loaded words words that trigger an emotional response or strong association by their use Avoid ambiguous words words that can be interpreted in more than one way Dont use double negative questions questions that can ask people to disagree with a negative question

Constructing Questions
How to ask questions

Dont use leading questions questions that encourage the respondent to answer in a certain way, typically by indicating which is the right or correct answer Avoid threatening questions, or questions that make respondents afraid or embarrassed to give an honest answer

Constructing Questions
How to ask questions Dont use double-barreled or compound questions questions that ask two or more questions in one Ask question in the language of your respondents, using the idioms and language appropriate to the samples level of education, vocabulary of the region, etc. Review Box 9.9 Examples of poorly worded questions and the suggested revisions

Constructing Questions
How to ask questions

Question order & context

Responses to questions can be affected by the question order as earlier questions provide a context for later ones and people may try to be consistent in their answers to questions on the same topic.

Constructing Questions
1. 2.

How to ask questions

Question order & context

3.

Consider a logical order that makes participation easy To encourage participation start with interesting, no threatening questions, and save questions about sensitive topics for the middle or end Conclude with cool down to minimize discomfort of participant

Summary
Considerations when choosing a data

collection method

Quiz Question 1
Open-ended questions allow for respondents to answer
a. b.

c.
d. e.

in their own words. according to pre-determined categories. as often as they wish. skip contingency questions none of the above

Quiz Question 2
If close-ended questions are used, it is important that the responses are
a. b.

c.
d. e.

mutually exclusive. exhaustive. created by respondents. a and b All of the above

Quiz Question 3
Which type of survey tends to have the highest response rate?
a. b.

c.
d. e.

Mailed questionnaire Phone interview In-person interview All three tend to yield high response rates None of these yield high response rates