Surveys

What is a Survey?
 Why Survey?
Get lots of structured information from lots of
people.
Simplify and standardize data collection.
Use statistics to make predictions.
Find out about things that can not be observed.
Such as Thoughts, Emotions, Opinions,
Intentions, Attitudes
The Need to Know Why

why people do or do not do something.

Likes, dislikes, attitudes, behaviors, influences
The Need to Know How
the process consumers go through before taking
some action.
Decision making process

The Need to Know Who

Information on age, income, occupation, marital status,
stage in the family life cycle, education, and other lifestyle
factors is necessary to the identification and definition of
market segments.
 Survey Methods
Personal Interview
Door-to-Door (in-home), Mall-Intercept, Purchase-Intercept,
Executive (office)
Telephone-Interview
People Computer assisted (CATI)
Self-Administered
Mail Survey
Fax
One-time vs. Panels
Computer
Direct Computer Interviewing
Interactive voice response technology-Automated Telephone
Surveys
Internet
 A Classification of Survey Methods
SURVEY
METHODS

TELEPHONE FACE TO FACE MAIL COMPUTER

TRADITIONAL CATI POSTAL POSTAL

TELEPHONE SURVEY PANEL

FAX E-MAIL INTERNET

IN HOME MALL CAPI
INTERCEPT

DIRECT COMPUTER
INTERVIEW
Personal Interviews
 Personal Interviews
Advantages
Can arouse and keep interest
Can build rapport and enlist cooperation
Ask complex questions
Immediate feedback
Can use visual and other aids
Clarify misunderstandings
High degree of flexibility
Probe for more complete answers
Do not need an explicit or current list of households or
individuals
 Personal Interviews
Disadvantages:
Bias of Interviewer
Response Bias
Embarrassing/personal questions
Time Requirements
Cost Per Completed Interview Is High.
High refusal rate
A trained staff of interviewers that is geographically near the sample
is needed.
The total data collection period is likely to be longer than for most
procedures.
High selection error non-representative sample
 Telephone Interviewing
What are the Important Aspects of Telephone
Interviewing?

Selecting telephone numbers

Pre specified list
A directory
Random dialing procedure
Random digit dialing
Systematic random digit dialing (SRDD)
The introduction
When to call
Call reports
 Telephone Interviewing
Advantages:
Central location, under supervision, at own hours
More interviews can be conducted in a given time
Travelling time is saved
More hours of the day are productive
Repeated call backs at lower cost
Lower administrative costs/ Lower cost per completed
interview
Less sample bias
Better access to certain populations
Shorter data collection periods.
 Telephone Interviewing
Limitations and Disadvantages:
Inability to employ visual aids or complex tasks
Can't be longer than 5-10 min. or they get boring
Amount of data that can be collected is relatively less
A capable interviewer essential
Sample bias -- Not all people have phones, or are not listed
Nonresponse associated with RDD sampling is higher than with
interviews
Possibly less appropriate for personal or sensitive questions if no
prior contact
National Do Not Call List 1-866-580-DNCL (866-580-3625)
http://www.LNNTE-DNCL.gc.ca
 Increasing Phone Survey Response
Call at a convenient time (Weekdays 7-9
PM, Sunday afternoon)
Have a nice Pleasant introduction
Emphasize you are not selling anything.
State how long it will take.
Keep the survey short
Self-Administered Types of Survey Research

Direct
Direct Mail
Mail Survey
Survey
Questionnaire
Questionnaire is is distributed
distributed to
to and
and
returned
returned from
from respondents
respondents viavia the
the
postal
postal service.
service.
Respondent
Respondent
Reads
Reads
Survey
Survey Mail
Mail Panel
Panel Survey
Survey
Questions
Questions Selected
Selected group
group ofof individuals
individuals that
that have
have
made
made anan advance
advance agreement
agreement toto
and
andRecords
Records participate in a series of direct mail
participate in a series of direct mail
Answers
Answers surveys.
surveys.
Without
Without
Assistance
Assistance
Drop
Drop Off
Off Survey
Survey
Questionnaires
Questionnaires areare left
left with
with respondent
respondent
to
to be
be completed
completed at at aa later
later time
time and
and
returned
returned to
to the
the researcher.
researcher.
 Mail Surveys
Requires a broad identification of the individuals to be
sampled before data collection begins
Ad Hoc Mail Surveys (cold):
Questionnaires for a particular project sent to selected names
and addresses with no prior contact by the researcher.
Mail Panels (warm):
Pre-contacted and screened participants who are periodically
sent questionnaires.
A mail panel is a type of longitudinal study. A longitudinal study
in one in which the same respondents are re-sampled over time.
 Mail Surveys
Some Decisions That Need to Be Taken
Type of Return Envelope
Postage
Method of Addressing
Cover Letter
The Questionnaire Length, Layout, Color,
Format Etc
Method of Notification
Incentive to Be Given
 Mail Surveys
Advantages

Relatively low cost

Reliable answers as no inhibiting intermediary
Survey answered at respondents discretion
Can be accomplished with minimal staff and facilities.
Provides access to widely dispersed samples.
Respondents have time to give thoughtful answers,
look up records, or consult others.
 Mail Surveys
Disadvantages
No control over whom the respondent consults before answering the questions
The identity of the respondent is inadequately controlled
The speed of the response can't be monitored
No control on the order in which the questions are exposed or answered
Open questions usually are not useful.
Good reading and writing skills are needed by respondents.
The interviewer is not present to exercise quality control with respect to
answering all questions, meeting questions objectives, or the quality of
answers provided.
High non-response rate
 Mail Surveys
Disadvantages (Contd.):
The respondent may not clearly understand the
question and no opportunity to clarify
No long questionnaires
Need for good up-to date mailing list
Response rate is generally poor
Number of problems such as obsolescence,
omissions, duplications, etc
Ineffective as a way of enlisting cooperation.
 Factors Affecting the Response Rate
Perceived amount of work required, and the
length of the questionnaire
Intrinsic interest in the topic
Characteristics of the sample
Credibility of the sponsoring organization
Level of induced motivation
 Tactics Employed to Increase Mail
Survey Response Rate
Advance postcard or telephone call alerting respondent of
survey.
Follow-up postcard or phone call.
Monetary incentives (how much?).
Premiums (pencil, pen, keychain, coupons, etc.).
Postage stamps rather than metered envelopes.
Self-addressed, stamped return envelope.
Personalized address and well-written cover letter.
Personally signed cover letter.
 Tactics Employed to Increase Mail
Survey Response Rate
Entry into drawing for prize.
Emotional appeals.
Affiliation with institutions or reputed organizations.
Multiple mailings of the questionnaire.
Bids for sympathy.
Offer to share information from the survey.
Promise of contributions to favorite charity.
Drop-off questionnaires
The interviewer can explain the study, answer
questions, and designate a respondent.
Response rates tend to be like those of personal
interview studies.
There ismore opportunity to give thoughtful
answers and consult records.
Costs about as much as personal interviews.
A field staff is required.
 Fax Surveys
Advantages

Relatively low cost

Can be accomplished with minimal staff and
facilities
Provides access to widely dispersed samples.
Respondents have time to give thoughtful answers.
Local faxes are free.
Administrative costs are fixed.
It is fast.
List management is easy.
Can send and receive by computer.
 Fax Surveys
Disadvantages

Higher fixed costs for computer/fax equipment,

multiple phone lines.
Cost varies by time on line, time of day, distance,
and telephone carrier.
Generally limited to organizational populations.
Loss of anonymity.
 Internet Surveys
To realize the importance
of the marketing research
Internet Samples interviewer.
Unrestricted
Open to any Internet user.
Screened
adjust for unrepresentitiveness of the self-selected
respondents by imposing quotas based on some desired
sample characteristics

Recruited
to target populations in
surveys that require more
control of the sample
Greenfield
 Internet Surveys
Advantages:
Theadvantages of interviewer administration (In contrast to
mail surveys).
smaller staff needed,
High-speed,

Instantaneous data access real time reporting

Cost efficient
Automatic data entry
Multimedia stimuli
Easy to update
Ability to reach a lot of people, and often those hard to reach
 Internet Surveys
Advantages:
Potential for longitudinal studies
Surveys can be unobtrusively included with a general
site
pre-screening of respondents possible
tracking
Easy to personalize
No geographic boundaries
supervision and quality control potentially better.
better response rate from a list sample than from mail
 Internet Surveys
Disadvantages:
Internet
users are not representative of the population
as a whole (strong sample bias)
Strongselection bias for respondents who are not pre-
screened
security/privacy issues
unrestricted: anyone can complete the questionnaire
fully self-selecting
 Free Trial Survey Software

Free Online Surveys

eSurveypro

Question Pro

Keysurvey

Checkbox survey
E-mail Questionnaires
Thequestionnaire is prepared like a simple
E-mail message, and is sent to a list of known
E-mail addresses. The respondent fills in the
answers, and E-mails the form plus replies
back to the research organization
Increasing Response: Internet/E-mail Survey

Almost all the actions listed for mail survey

should hold for E-Mail/Internet survey with
some modifications.
The questionnaire should be fairly short.
No need for advance notification
Have short, pleasant introduction
Monetary incentives not given. But sharing
information would be nice.
Reminder e-mail would be useful.
 Criteria for Determining Choice Of Particular Survey Methods
Sample Criteria: Ability to reach and get responses from the desired sample
(sample control)
right type of people
adequate sample size.
Information Criteria: Ability to get the desired information from respondents.
Need to Expose Respondents to Various Stimuli or Perform Certain Specialized Tasks e.g. Taste tests,
product concept and prototype tests, etc.
Length of Questionnaire
Degree of Structure of the Questionnaire
Control social desirability
Administrative Criteria:
time for data collection and analysis
Interviewer control
Error control
Budget
 Factors Determining the Choice of
Survey Method.
Personal:
Use for long, complex questionnaires where respondent is
important and budgets are high.
Mail:

Use when you have a mailing list, somewhat long / complex

questionnaire, and budgets are low.
Phone:

Use to reach a large number of people quickly and you have a

E-Mail/Internet:.
short questionnaire that can be easily understood.

Use when target audience is educated, topic interesting, short

questionnaire, have e-mail list, representativeness not a major
 Comparative Evaluation
Personal Phone Mail Internet
Sample control
Use of physical stimuli
Diversity of questions
Length
Perceived anonymity
Potential for interviewer bias
Field force control
Speed
Cost
 Advantages & Disadvantages of Survey
MethodsPersonal Mail/Self Report Telephone Internet
Speed of Moderate to fast No control over Very fast Fastest
data return of
collection questionnaire
Geographic Limited to High Good Best
flexibility moderate
Respondent Excellent except Moderate poorly Good Best
cooperation in shopping designed
malls questionnaires
have poor response
rates
Versatility of Very versatile Highly Moderate Good
questioning standardized
format
Advantages & Disadvantages of Survey
Methods
Personal Mail/Self Report Telephone Internet
Questionnaire Long Varies depending Moderate/ Moderate
length on incentive Short
Respondent Lowest Highest Moderate Moderate
misunderstanding
Interviewer High None Moderate None
influence
Supervision of Moderate Not applicable High None
interviewers

Anonymity of Low in face- High Moderate Low

respondent to-face
situations
 Advantages & Disadvantages of Survey
Methods
Personal Mail/Self Report Telephone Internet
Ease of Difficult Easy, but takes Easy Easy
callback or time
follow-up
Cost Highest Low Low to Lowest
moderate
Special Visual materials Respondents may Field and Data
features may be shown answer questions supervision of collection
or at own data collection automatic
demonstrated; convenience; has are simplified; Skip patterns
extended time to reflect on quite automatic
probing answers adaptable to
possible computer
technology

Note: These reflect typical situations. For example, an elaborate mail survey may be
far more expensive than a short interview, but this is generally not the case.
 Potential Problems with Surveys
How accurate are the results?
Did you ask the right questions in the right way?
Did you ask the right people?
Did they tell you the truth, the whole truth, and
nothing but the truth?
Will they?
Can they? Do they have the knowledge, opinions, attitudes, or facts
required. Do they understand the questions?
Structure causes a loss of data richness.
Have the Respondents/Interviewers understood and
correctly recorded the responses
Lack of control causes time and response problems.
 Overview of the Types of Errors in
Survey Research Methods
Survey research errors can be classified as either:
Random Sampling Error Nonsampling Error
statistically measured difference (Systematic Error)
between the actual sampled results from mistakes or
results and the estimated true problems in the research design
population results. or its execution
error because of chance variation Causes data to be misleading or
Reduce by increasing sample size
incomplete in some systematic
way from true population
Range can be estimated with a parameter
certain level of confidence If you are aware of the problem
you may be able to deal with it.
 Systematic Sample Design Error
Frame Error: The list from which you draw your sample is not what
you think it issome on it dont belong and/or some who belong are
not on it sample is not representative of population (e.g. telephone
directory)
Population Specification Error: incorrect definition of the universe or
population from which the sample is to be selected. You left people out
of the study.
Selection Error: You include or exclude people in the sample so that it
is not random. Use of incomplete or improper sampling procedures or
when appropriate selection procedures are not properly followed (e.g.
non-probability samples such as intercept surveys)
 Systematic Measurement Error
Difference between the information sought and that obtained

Surrogate Information Error

Acquisition of the wrong data because wrong question was asked.
failure to define problem, e.g. New Coke
Interviewer Error
interviewer may, consciously or unconsciously, influence respondents to
give untrue or inaccurate answers. training problem
Instrument Bias
misunderstood or leading questions questionnaire design
Processing Error
sloppy data input
Non-response Bias
differences between the did vs. did not answer question
Response Bias
a failure to tell the truth, consciously or unconsciously
Your company has just launched a
new brand of pancake syrup. In
order to improve sales the marketing
department would like to know why
consumers buy the brands and sizes
they do. How would you go about
collecting this information?
Calgary Transit would like to
conduct a ridership survey. Funds
are limited but they need a
relatively large sample. How
would you suggest they gather the
information?
Your company makes the new flip and fold device.
You want to find out what consumers think about it
and how much they are willing to pay. What sort of
survey technique do you use and why. Who do you
target in your survey
 OBSERVATIONAL RESEARCH

Watching what people do

The information must be observable

Helpful if the behavior is repetitive and of short duration

Approaches to observational research

Natural Versus Contrived Situations
Open Versus Disguised Observation
Structured Versus Unstructured
Human Versus Machine Observers
What does gathering data through
Observation entail

What are the advantages of observational

data collection as opposed to surveys?

What are the relative disadvantages of

observational data collection as opposed to
surveys?
 Main Observational Research Methods
Direct Observation
Shopper Patterns and Behavior
Contrived Observation
Mystery Shopper
Content Analysis
Analyzing written material into meaningful units, using
carefully applied rules
Physical Trace Measures
Garbology
Ethnographic Research
The researcher becomes Immersed in or part of the group
Behavior (Emotion) Recording Devices
Toothpaste manufactures have found consistently that if they ask for
detailed information on the frequency with which people brush their
teeth, and then make minimal assumptions as to the quantity of
toothpaste used on each occasion, as well as spillage and failure to
squeeze the tube empty, the result is a serious overstatement of
toothpaste consumption.
How would you explain this phenomenon?
Would it be possible to design a study to overcome these
problems and obtain more accurate estimates of consumption?
Advantages:

We see what people actually do

May avoid interviewer bias

Disadvantages:

No information on motives attitudes or

intentions
Time-consuming and expensive
 Machine observation
Traffic Counters
Time and flow in retail stores
Behavior Measurement
People Reader: reading habits
Physiological Measurement
EEG: electroencephalogram
GSR: galvanic skin response
Pupilometer: pupil dilation
Scanner Based
Store scanners read the UPC codes on products and
produce instantaneous information on sales