QUESTIONNAIRE DESIGN

Patrick S. Romano, MD, MPH

Professor of Medicine and Pediatrics

Questionnaire Design
1. List variables
Sources include patients/subjects (focus groups, key informant interviews), clinical observation, theory or conceptual framework, prior research, and expert opinion.

2. Borrow from other instruments

Save development effort (reinventing the wheel) Borrow reliability, validity, variance estimates Facilitate comparison with previous studies

3. Solicit input from colleagues and friends

Questionnaire Design (cont)

4. Prepare a draft 5. Circulate and revise 6. Pretest 7. Shorten and revise again

Method of Administration
Advantages of verbal interview:
Interviewer can clarify unclear questions Literacy is not required

Interviewer can collect more complex answers and observations

Interviewer can minimize missing and inappropriate responses Interviewer can prevent respondent from answering out of sequence

Method of Administration (cont)

Advantages of written questionnaire:
Much less staff time is required, with no dead time waiting for potential participants
Less potential for observer bias Anonymity may minimize social desirability bias, and encourage more honest responses to sensitive questions

Computer-assisted telephone interviewing (CATI) by random-digit dialing (RDD) combines advantages of both verbal and written methods.

Questionnaire Design: General Principles

Open-ended vs closed-ended questions:
Open-ended questions generate answers that are more nuanced and information-rich Open-ended questions generate answers that are more difficult to categorize and analyze

Questionnaire Design: General Principles (cont)

Closed-ended questions are easier
to interpret, with lower respondent burden.

Developing a sufficiently specific but

exhaustive list of response options may be very difficult.

Closed-ended questions may lead

the respondent in an inappropriate direction.

Questionnaire Design: General Principles (cont)

Length:

No longer than absolutely necessary (24

pages)

Use branching questions and skip patterns

to reduce length

Avoid deceptively short questions with

high respondent burden (complex tables, rank ordering, mental calculations, check all that apply)

Questionnaire Design: General Principles (cont)

Visual design:
Attractive, uncluttered format with a professional appearance and consistent graphic design
Consider light pastel colors, illustrations, 2 columns Avoid excessively small or unusual fonts Number and carefully align (vertically) questions and response options Avoid loose pages; booklet format if possible. Avoid splitting questions across columns/pages. Minimize the number and abruptness of format changes; use transitional sentences.

Questionnaire Design: General Principles (cont)

Sequence:
Frame the survey briefly but carefully
Place instructions where needed, not just at the beginning

Start with benign, easy, salient questions (consistent with cover letter)
Bury most sensitive questions about personal behaviors

Threatening demographic questions near end End with thank you and follow-up information

Questionnaire Design: General Principles (cont)

Facilitating response:
Design survey form to fold on itself or include
envelope

Self-addressed, stamped (not metered) return

Prepaid incentive ($5-10 cash best) Signed endorsement/cover letter(s)

Prenotification letter, reminder postcard, followup mailings and/or telephone calls

Questionnaire Design: General Principles (cont)

Facilitating data entry and follow-up:

Precode closed-ended questions

(avoid negative scale values)

Number questions consecutively

Identification number

Phrasing Questions
1. Clarity/specificity Avoid nonspecific response options, but also
avoid seeking unachievable specificity. Often? Sometimes? Regularly?

2. Simplicity Avoid complex or technical phrasing. Use complete but short sentences (avoiding
double negatives and redundancy between question stem and response options)

3. Neutrality Avoid loaded questions, arguable prefatory

statements, and judgmental words/phrases

Phrasing Questions (cont)

4. Sensitivity Encourage unorthodox and socially undesirable" responses (depersonalization): Everybody does it How many times Many experts say responses Did you happen to Many people have trouble remembering

Discourage over reporting of socially desirable

Allow for either agreement or disagreement in

question stem Funnelling questions to introduce sensitive topics

Phrasing Questions (cont)

5. One topic per question Avoid double-barreled questions
Do African-Americans and Latinos suffer from discrimination

6. Specify an appropriate time frame

Typical/usual versus maximal/minimal Depends on salience of topic

7. Improving recall
Aided recall (memory cues, prompts)

Bounded recall (time window) Records or diaries

Phrasing Questions (cont)

8. Mutually exclusive and exhaustive response options
Use of other fields

9. Consider question polarity and sequence (survey as conversation)

Avoid yea-saying patterns and carryover effects, but maintain consistency when possible

10. Check for technical accuracy

Scaling Responses
Visual analogue (thermometer)
scales Frequency-weighted multisymptom indices
I-PSS score

Multi-symptom checklists
Charlson/Elixhauser comorbidity lists

Grading or ranking

Scaling Responses (cont)

Likert (summative) scales
Add ordinal scores for multiple items
measuring agreement or disagreement with stated attitudes or beliefs.

Number of symmetric categories (4-10) Include or omit middle category (or put no
opinion at end of response options)

Consider ceiling and floor effects Watch for over-endorsement of labeled

boxes

Scaling Responses (cont)

Adjectival scale (variation of Likert) Semantic differential scale
Anchoring the extremes is usually desirable to increase variation Labeling intermediate points encourages endorsement of those points

Guttman (cumulative) scales

Add the number of ordered statements
eliciting agreement by the respondent Ordering errors may be problematic (coefficient of scalability)

Thank You !