McGraw-Hill/Irwin Copyright © 2009 by The McGraw-Hill Companies, Inc. All rights reserved.

Chapter
Chapter 13
13
Questionnaire
Questionnaire Design:
Design:
Concepts
Concepts and
and Issues
Issues
 Learning Objectives

1. Describe the process of questionnaire

design.
2. Discuss questionnaire development
using the “flowerpot” approach.
3. Summarize the characteristics of
good questionnaires.
4. Understand the role of cover letters.
5. Explain the importance of other
documents used with questionnaires.

13-3
Questionnaire
.... ..aaformalized
formalized
framework
frameworkconsisting
consistingofof
aaset
set of ofquestions
questionsand
and
scales
scalesdesigned
designedto to
generate
generateprimary
primary data.
data.

Questionnaire
Questionnaire construction
construction .. .. .. involves
involves taking
taking
established
establishedsets
setsof
of measurement
measurement scales scalesand
and
formatting
formatting them
them into
intoan
aninstrument
instrumentfor forcollecting
collecting data.
data.
Good
Goodquestionnaires
questionnairesfacilitate
facilitatereliable
reliableand andvalid
validdata.
data.
13-4
 Questionnaire
Questionnaire –– Theoretical
Theoretical
Components
Components

Words
Words

Questions
Questions Formats
Formats

Hypotheses
Hypotheses
Questionnaire
Questionnaireformat
format––layout
layoutof
ofquestions
questions//
measurement
measurementscales
scalesinto
intoaasystematic
systematicinstrument.
instrument.
13-5
 Theoretical Principles
of Questionnaire Design

Questionnaire Design Weaknesses

 Process should be scientific, integrating
established rules of logic, objectivity, and
systematic procedures.
 Multiple words create questions and
questions combine to form
questionnaires, but writing questions
does not result in a good questionnaire.

13-6
 Questions / Setups

Question
Question Format
Format
•• Structured
Structured
Question •• Unstructured
Unstructured
Phrasing
Issues
Question
Question Quality
Quality
•• Good
Good
•• Bad
Bad 7

13-7
 Theoretical Principles
of Questionnaire Design

Question format – unstructured questions

= open-ended approach . . .
 respondents reply in their own words.
 require more thinking on the part of the
respondent.
 format of open-ended questions depends
on the data collection method – personal
interviews, computer-assisted telephone
interviews, or online and offline self-
administered surveys.
13-8
 Question Issues . . .
Type
Typeof ofQuestion
QuestionFormat
Format
 Unstructured Questions––
UnstructuredQuestions
oopen-ended
pen-endedformat
formatwhere
where
respondent
respondentreplies
repliesin
intheir
theirown
own
words.
words.
 Structured Questions––
StructuredQuestions
cclosed-ended
losed-endedformat
formatwhere
where
respondent
respondentchooses
choosesfrom
fromaa
predetermined
predeterminedset
setof
ofpossible
possible
responses. Quality
responses. Qualityof
ofQuestions
Questions
 Wording
Wording––could,
could,should,
should,
or
orwill
will??
??
 Bad
BadQuestions
Questions

 Lack
LackUnderstanding
Understanding

 Unanswerable.
Unanswerable.

 Leading
Leadingor
orloaded.
loaded.

 Double-barreled.
Double-barreled.
13-9
Examples of Unstructured
Question/Setup Designs

13-10
 Structured
Structured questions
questions
(closed-ended
(closed-endedformat)
format)

Respondent
Respondentchooses
choosesaaresponse
responsefrom
fromaapre-
pre-
determined
determinedset
setof
ofalternatives.
alternatives.

Popular
Popularformat
formatin
inmost
mostself-administered
self-administered
questionnaires.
questionnaires.

More
Moreopportunities
opportunities to
tocontrol
controlthe
thethinking
thinking
respondents
respondentsdo
dowhen
whenanswering
answeringquestions.
questions.

Interviewer
Interviewerbias
biaseliminated.
eliminated.

13-11
Examples of Structured
Question/Setup Designs

13-12
 Incomprehensible
Incomprehensible

Bad
Bad
questions
questions .. .. .. Unanswerable
Unanswerable
prevent
prevent oror
distort
distort
communication
communication
between
between Leading
researcher Leading // loaded
loaded
researcher andand
respondent.
respondent.

Double-barreled
Double-barreled

13-13
 Question quality – bad questions
defined . . .
• Incomprehensible – wording, concept or both
cannot be understood.
• Unanswerable – respondent does not have the
information needed, or none of the answer choices
apply to the respondent.
• Leading or loaded – respondent is forced or
directed into a response that would not ordinarily
be given if all possible response categories or
concepts were provided.
• Double-barreled questions – address more than
one issue at a time.
13-14
 Hypotheses relate to . . .

 Nature
Nature of
of respondent.
respondent.

 Relationship
Relationship between
between expressed
expressed attitudes
attitudes and
and
behavior
behavior ofof the
the respondent.
respondent.

 Sociological
Sociological structures
structures and
and their
their influences
influences on on
the
the respondent.
respondent.

 Meaning
Meaning of of words
words and
and respondent’s
respondent’s grasp
grasp ofof
language
language and/or
and/or concepts.
concepts.

 Relationships
Relationships among
among respondent’s
respondent’s knowledge,
knowledge,
attitudes,
attitudes, and
and marketplace
marketplace behaviors.
behaviors.

 Descriptive
Descriptive andand predictive
predictive capabilities
capabilities ofof the
the
constructs.
constructs.

13-15
Examples of Different Types of
Hypotheses Used in Information
Research

13-16
 Flowerpot
Flowerpot approach
approach
.. .. .. framework
framework for
for integrating
integratingsets
setsofof
question/scale
question/scale measurements
measurementsinto intoaalogical,
logical,
smooth-flowing
smooth-flowing questionnaire
questionnaire

Identifies
Identifiesrules-of-thumb
rules-of-thumband
anddecision
decisionfactors
factors
regarding
regarding ......
•• Construct
Constructdevelopment
development
•• Attributes
Attributesororobjects
objects
•• Question/measurement
Question/measurementscale
scaleformats
formats
•• Wording
Wordingof ofquestions
questions
•• Scale
Scalepoints
points

13-17
 Questionnaires –
Flowerpot
Approach
Reduces biased data Question Layout

Data requirements are Flow of data/questioning

determined for each is from general to more
objective with most specific information.
general data going into
the biggest flowerpot
and the next most
general data going into
a smaller pot.

13-18
Steps in the Development of
Survey Instruments

13-19
The Flowerpot Approach

13-20
 Flowerpot Approach
Construct
Construct development
development

Object
Object attributes
attributes

Helps
Helps
researchers
researchers Question/scale
Question/scale formats
formats
decide
decide .. .. ..

Question
Question wording
wording

Number
Number of
of scale
scale points
points
13-21
 Flowerpot Approach

Guidelines . . .
 Questionnaires begin with an introduction
section.
 Researchers must decide how many research
objectives are to be examined.
 Specific information is then collected for each
of the objectives.
 Questionnaires end with demographic and
socioeconomic questions.
13-22
 Flowerpot Approach to Questionnaire
Development

Developing a Flowerpot-Designed
Questionnaire . . .
Step 1 – Transform research objectives into
information objectives
Step 2 – Determine appropriate data collection
method
Step 3 – Determine information requirements
for each objective
13-23
 Flowerpot Approach to Questionnaire
Development

Step 4 – develop specific question &

measurement scale formats
Key decisions . . .
Type of data
Question/scale format
Question wording
Step 5 – Evaluate questions &
measurement scales

13-24
Develop Specific Question/Scale
Measurement Formats

13-25
 Develop Specific
Question/Scale Measurement
Formats (continued)

13-26
 Develop Specific Question/Scale
Measurement Formats (continued)

13-27
 Develop Specific Question/Scale
Measurement Formats (continued)

13-28
Guidelines for Evaluating the
Adequacy of Questions

13-29
 Flowerpot Approach to Questionnaire
Development

Step 6 – Establish questionnaire format

and layout
• This step becomes the core of the Flowerpot
approach
Step 7 – Evaluate the questionnaire and
layout
• Questionnaire layout is reviewed and
evaluated according to objectives
Step 8 – Obtain the client’s approval
• May have new information or concerns –
creating need for modifications
13-30
Flowerpot Approach to Questionnaire
Development

Step 9 – Pretest and revise the

questionnaire
• Based on individuals representing the
final respondents
Step 10 – finalize questionnaire
format
Step 11 – Implement survey
• Begin the collection of data
13-31
Considerations in Questionnaire
Design

13-32
Role of a Cover Letter
Initial
Initial Contacting
Contacting Device
Device

Encourages
Encourages Participation
Participation

Provides
Provides Study
Study Information
Information

Communicates
Communicates Study’s
Study’s
Legitimacy
Legitimacy

Improves
Improves Response
Response Rates
Rates

13-33
Guidelines for Developing Cover
Letters

13-34
Supplemental Documents
Associated with Survey
Instruments

 Supervisor
Supervisor instructions
instructions

 Interviewer
Interviewer instructions
instructions

 Screening
Screening forms
forms

 Quota
Quota sheets
sheets

 Data
Data sheets
sheets

 Rating
Rating cards
cards

 Call
Call record
record sheets
sheets
13-35
Example of Supervisor Instructions
for a Retail Bank Study Using
Personal Interviews

13-36
Example of Interviewer Instructions
for a Retail Bank Study Using
Personal Interviews

13-37
 Example of the Question/Scale
Format and Rating Card Used in
Collecting Data in a Retail Banking
Survey

13-38