Questionnaire Design

Goa Institute of Management 2012

Goa Institute of Management 2012

Information Requirement
To determine how often consumer purchases pizza from Papa Johns.

Goa Institute of Management 2012

Goa Institute of Management 2012

 A researchers ability to design a good scale is, by itself not enough to guarantee that the appropriate data will automatically be collected.

Goa Institute of Management 2012

 Great Weaknesses of Questionnaire Design

Theory

many researchers do not understand the theory that underlies questionnaire development
Process itself should be a scientific one that integrates established rules of logic, objectivity, and systematic procedures Words go into questions and that questions go into questionnaires, but not everyone understands that writing questions does not give you a questionnaire Administrative process Response Strategy

Goa Institute of Management 2012

 A questionnaire is a formalized set of questions for obtaining information from respondents. It must translate the information needed into a set of specific questions that the respondents can and will answer. A questionnaire must uplift, motivate, and encourage the respondent to become involved in the interview, to cooperate, and to complete the interview. A questionnaire should minimize response error.
Goa Institute of Management 2012 8

Maximising the response rate

If you were sending out a questionnaire, what would you do to maximise the response rate?

Goa Institute of Management 2012

The basic functions of questionnaire

It translates research objectives to be questions It standardizes the questions and the response categories: every respondents can give the information that we want It keeps communicating with respondents in a professional way (cooperation and motivation). It is the permanent records of survey It is helpful for the process of data analysis It is reliable information provided by respondents
Goa Institute of Management 2012 10

Advantages of questionnaires
Can reach a large number of people relatively easily and economically

Provide quantifiable answers

Relatively easy to analyse

Goa Institute of Management 2012

11

Disadvantages of questionnaires
Provides only limited insight into problem
Limited response allowed by questions Maybe not the right questions are asked

Varying response
Misunderstanding/misinterpretation

Need to get it right first time

Hard to chase after missing data

Goa Institute of Management 2012

12

Questionnaire design process

Goa Institute of Management 2012

13

Clear specification

Goa Institute of Management 2012

14

Questionnaire relevancy
No unnecessary information is collected and only information needed to solve the problem is obtained. Be specific about your data needs; tie each question to an objective

Goa Institute of Management 2012

15

 Theoretical Components of a Questionnaire (or Data Collection Instruments)

Questionnaires
Words
to use in creating the questions and scales for collecting raw data from respondents Wording Problems

Goa Institute of Management 2012

16

 Understanding of the questions Appropriateness of primary scale descriptors Balanced versus unbalanced scales Forced or free choice scales

Goa Institute of Management 2012

17

Question Content
Should this question be asked?
Is the question of proper scope and coverage? Can the participant adequately answer this question as asked?

Will the participant willingly answer this question as asked?

Goa Institute of Management 2012

18

 Type of question format

Unstructured questionsopen-ended, the respondents reply in their own words
Requires more thinking on the part of the respondent Format of open-ended questions depend on the data collection methodpersonal interviews, traditional, and computer-assisted telephone interviews, or online and offline self-administered surveys

Goa Institute of Management 2012

19

 Structured questionsclosed-ended formatted questions, where the respondent provides a response from a predetermined set of possible responses.
Popular format in most self-administered types of questionnaires Provides the researcher greater opportunities to control the thinking that respondents do in order to answer a question

Interviewer bias eliminated

Goa Institute of Management 2012

20

Question Phrasing
Need to decide how what is to be asked should be phrased Alternative question formats are shown below

Each format has its advantages and disadvantages

Open ended Simple dichotomy Multiple choice

Question type

Closed ended

Determinant choice
Goa Institute of Management 2012

Check-list
21

Alternative Question Formats

1. Open-ended Questions in which the respondent answers in his own words 2. Closed-ended (or Fixed Alternative) Question in which respondent selects one or more options from pre-determined set of responses
Simple dichotomy Closed ended question with only two response alternatives Multiple Choice Closed ended question with more than two response alternatives

Determinant choice Multiple choice question in which respondent must select only one of the response alternatives Checklist question - Multiple choice question in which respondent can select more than one of the response alternatives
Goa Institute of Management 2012 22

 Sometimes, several questions are needed to obtain the required information in an unambiguous manner. Consider the question,
Do you think Coca-Cola is a tasty and refreshing soft drink? (Incorrect)

Such a question is called a double-barreled question, because two or more questions are combined into one. To obtain the required information, two distinct questions should be asked:
Do you think Coca-Cola is a tasty soft drink? and Do you think Coca-Cola is a refreshing soft drink? (Correct)

Goa Institute of Management 2012

23

Overcoming Inability To Answer

Can the Respondent Remember?
How many gallons of soft drinks did you consume during the last four weeks?

(Incorrect)

How often do you consume soft drinks in a typical week? (Correct) 1. ___ Less than once a week 2. ___ 1 to 3 times per week 3. ___ 4 to 6 times per week 4. ___ 7 or more times per week
Goa Institute of Management 2012 24

Overcoming Unwillingness To Answer

Legitimate Purpose
Explaining why the data are needed can make the request for the information seem legitimate and increase the respondents' willingness to answer.

Sensitive Information
Respondents are unwilling to disclose, at least accurately, sensitive information because this may cause embarrassment or threaten the respondent's prestige or self-image.
Goa Institute of Management 2012 25

Choosing Question Wording

Use Ordinary Words
Do you think the distribution of soft drinks is adequate? (Incorrect)
Do you think soft drinks are readily available when you want to buy them? (Correct)

Goa Institute of Management 2012

26

In a typical month, how often do you shop in department stores? _____ Never _____ Occasionally _____ Sometimes _____ Often _____ Regularly

In a typical month, how often do you shop in department stores? _____ Less than once _____ 1 or 2 times _____ 3 or 4 times _____ More than 4 times (Correct)

Goa Institute of Management 2012

27

Leading questions
Leading question: a question that suggests or implies a certain answer.

Causes:
The bandwagon effect e.g. Most Saudis have stopped eating junk food. Do you eat junk food? Partially mentioning some alternatives e.g. which fast food restaurant do you prefer, Al-Tazaj or others? Questions with the phrase: Dont you think that ..

Goa Institute of Management 2012

28

Requirements for Close-Ended Questions

Response categories provided for each close-ended question should be mutually exclusive and

exhaustive
1. Mutually Exclusive Response categories must be such that the same respondent cannot be classified into more than one category; e.g. the categories SR1,000-5,000 and SR5,000-10,000 are not mutually exclusive. 2. Mutually Exhaustive Response categories should include all possible response options. Sometimes this is achieved by including a response option like Other (Please specify).
Goa Institute of Management 2012 29

Typical problems in wording questions

Goa Institute of Management 2012

30

Question Sequence
Two related issues: 1. Order of questions in questionnaire. 2. Order of answer alternatives for specific questions. Both can lead to order bias.

Goa Institute of Management 2012

31

Determining the Order of Questions

Effect on Subsequent Questions
Q1: What considerations are important to you in selecting a department store? Q2: In selecting a department store, how important is convenience of location? (Correct)

Goa Institute of Management 2012

32

Sequencing of Questions
The Funnel Approach From general to specific questions From easy to difficult questions

Goa Institute of Management 2012

33

 Flowerpot Approach
Specific frameworkfor integrating sets of question/scale measurements into a logical, smooth-flowing questionnaire

Goa Institute of Management 2012

34

 Questioning the layoutin good questionnaire design, the directional flow of data will be from general to more specific information Why? A survey that begins with general information questions promotes the development of the necessary comfort zone between the two parties. When respondents feel comfortable, the question/answer exchange process goes smoothly and respondents are more thoughtful and honest with their responses. of Management 2012 Goa Institute

35

Goa Institute of Management 2012

36

 According to the Flowerpot concept

Questionnaire begins with an introduction section The researcher must decide how many research objectives are to be explored Specific information would then be obtained within each of the objective categories Good questionnaire designs end with demographic and socioeconomic questions
Goa Institute of Management 2012 37

 The Flowerpot Concept

primarily used to determine the appropriate sequential order of the question and scale measurements, it has a direct impact on several of the other developmental steps
Determining the Information Objectives Determining Information Requirements

Goa Institute of Management 2012

38

More design issues

Questionnaire length

Although favour clarity over conciseness

Question wording affects responses Pilot testing Visual appearance...

Goa Institute of Management 2012 39

Visual appearance
Q1. Are you (please tick) A home owner A tenant Living with parents Other (please specify): Q1. Are you (please tick) A home owner A tenant Living with parents Other (please specify) of Management Goa Institute

2012

40

List and discuss the 11 steps in the questionnaire development process, and tell how to eliminate come common mistakes in questionnaire design

Development of a Flowerpot-Designed Questionnaire

Step 1Transform research objectives into information objectives Step 2Determine the appropriate data collection method Step 3determine information requirements for each objective
Goa Institute of Management 2012 41

List and discuss the 11 steps in the questionnaire development process, and tell how to eliminate come common mistakes in questionnaire design

Step 4develop specific question/scale measurement formats

Key decisions
Type of data Question/scale format Question and specific point wording

Step 5Evaluate question/scale measurements

Goa Institute of Management 2012

42

List and discuss the 11 steps in the questionnaire development process, and tell how to eliminate come common mistakes in questionnaire design

Step 6Establish the Flowerpot format/layout

This step becomes the core of the Flowerpot approach

Step 7Evaluate the questionnaire and layout

Questionnaires layout should be reviewed and evaluated according to the objectives

Step 8Obtain the clients approval

May have new information or concernscreating need for some types of modifications

Goa Institute of Management 2012

43

List and discuss the 11 steps in the questionnaire development process, and tell how to eliminate come common mistakes in questionnaire design

Step 9Pretest and revise the questionnaire

Should come from people who represent the respondent

Step 10finalize the questionnaire format

Placed in final format

Step 11Implement the survey

Begin the collection of the required raw data

Goa Institute of Management 2012

44

Questionnaire Pre-testing
Pretesting Process
Seeks to determine whether respondents have any difficulty understanding the questionnaire and whether there are any ambiguous or biased questions.

Preliminary Tabulation
A tabulation of the results of a pretest to help determine whether the questionnaire will meet the objectives of the research.
Goa Institute of Management 2012 45

One of the most important aspects of designing a questionnaire is improving your Response Rate, which requires providing the respondent with the motivation to complete the questionnaire. So consider:
1. Increasing perceived rewards 2. Decreasing perceived costs 3. Establishing trust

Ways to increase perceived reward

Show positive regard Say thank you Ask them for advice Give a tangible reward Make the questionnaire interesting State an upcoming deadline
Goa Institute of Management 2012 47

Ways to decrease perceived cost

Avoid subordinating language Do not embarrass the respondent Make questions appear short and easy Avoid asking personal information (if it must be asked, use soft wording and inform about confidentiality)
Goa Institute of Management 2012 48

Ways to establish trust

Provide a token of appreciation Provide a sense of legitimate authority Make completing the questionnaire seem important

Goa Institute of Management 2012

49

Overall Flowchart for Instrument Design

Goa Institute of Management 2012

50