Getting Data for (Business) Statistics: Whats new? Whats next?

2/10/2012

Statistical picture of a country

Getting Data for Business Statistics

How do we get the data we need for business statistics? Yesterday, today, tomorrow

Data
In time Complete Correct

Respondent Parameters

Survey Parameters in and out of control

NSI

2/10/2012

Getting Data for Business Statistics

Over the years: 1. Yesterday:

2. Today:

ICES-I* 1993 ICES-II 2000 CASM** 1980s ICES-III 2007

Challenges and developments A few examples Whats next ?

3. Tomorrow

* International Conference on Establishment Surveys ** Cognitive Aspects of Survey Methodology

2/10/2012 3

Getting Data for Business Statistics

Yesterday
ICES-I (1993): 1. Surveying various branches of industry:
agriculture, energy, health care, trade, finance, education, manufacturing industry

2. Quality of business frames & sampling 3. Data analysis & Estimation 4. Data collection methodology:
data quality, registers, non-response, Q-design

 Stove-pipe approach  Single-mode survey designs

2/10/2012 4

External business factors Econ. climate Regulatory requirements Political climate

Internal business factors Policy Data Resources Market position Informant: Mandate Data knowledge Job priority

The survey: Topic Population and sample Sponsor / Survey organisation Resources Planning Authority/confidentiality

The survey design Single mode Questionnaire Data WE want Modes of data Paper collection Contact Letters: Mandatory strategy

Motivation Black box Decision to participate

Answering behaviour

Response
In time Complete Correct

Respondent burden
De facto Perception

Survey designs not coordinated: Stove-pipe approach

5 NSI

NSI

2/10/2012

A business

Getting Data for Business Statistics

Yesterday
CASM (started in 1980s; USA, Germany): Cognitive Aspects of Survey Methodology From simple stimulus-response model to modelling Question-Answer Process:
comprehension retrieval evaluation response

Pre-testing facilities

2/10/2012

Getting Data for Business Statistics

Today
ICES-III (2007): 1. Survey data collection methodology:
questionnaire design & pre-testing survey participation: non-response reduction, response burden, bias mixed-mode designs & e-data collection understanding the response process in buss

2. Using administrative data 3. Business frames & Sampling 4. Weighting, Outlier detection, Estimation & Data analysis
2/10/2012 7

Statistical picture of a country

External business factors Econ. climate Regulatory requirements Political climate

Internal business factors Policy Data Resources Market position Informant: Mandate Data knowledge Job priority Image

The survey: Topic Population and sample Sponsor / Survey organisation Resources Planning Authority/confidentiality

The survey design Questionnaire Modes of data collection Contact strategy

Motivation Black box Decision to participate

Answering behaviour

Response
In time Complete Correct

Respondent burden
De facto Perception

Register data

More than one survey

More than once In other ways: Registers EDI

NSI

2/10/2012

A business

8 NSI

Getting Data for Business Statistics

Over the years

General picture: 1993: Stove-pipe approach Single-mode designs Survey organisation is central 2000: 2007:
Transition Systematisation and

standardisation of methods Towards multi-source/mixed-mode designs Respondent is central: tailoring

2/10/2012 9

Getting Data for Business Statistics

The data collection design today

Challenges: Good statistics:
relevant more & integrated information faster

Less money Less compliance costs:

2/10/2012

providing data only once to government powerful computers, access to the internet
10

New technologies: Consequences for the data collection

Getting Data for Business Statistics

The data collection design today

Use of administrative data:
Coordination of definitions:
variables units timeliness

Quality of register data:

-

Data collection without questionnaires:

2/10/2012

EDI: XBRL GPS If other sources are not possible or insufficient Getting insight in the data collection process
11

Surveys: Process measurement and quality control

Getting Data for Business Statistics

The data collection design today

Surveys:
Sampling:
- controlling for overlap across surveys - controlling for rotation over time (survey holiday)  one statistical business register In order to avoid this:

2/10/2012

12

Getting Data for Business Statistics

The data collection design today

Surveys:
Sampling:
- controlling for overlap across surveys - controlling for rotation over time (survey holiday)  one statistical business register

Mode:
Mixed-mode designs: paper, internet, CATI Computer-assisted Customization (tailoring) Controlling the completion process (routing, checks)

Questionnaires for web data collection:

-

2/10/2012

13

Getting Data for Business Statistics

The data collection design today

Surveys:
Contact strategy: - Mixed-mode:
.. paper letters, brochures, telephone, .. e-mails, website information

- Message:
.. Cooperation = mandatory! .. What, how, who, when?

- Cooperation no longer taken for granted:

.. Motivating and stimulating respondents: . Cialdini: Compliance (persuasion) principles . Dillman: Social Exchange Theory

- Two-way communication via the internet

2/10/2012 14

Getting Data for Business Statistics

The data collection design today

Process measurement and quality control: Paradata process data:
- Macro paradata (survey process data):
.. Process summaries: response rates, timeliness of response, quality of response over time

- Micro paradata (process data at R level):

.. Completion process: audit trails

2/10/2012

15

Getting Data for Business Statistics

Macro paradata
Timeliness of response (Monthly Survey)
Online (e-mail + voor html-vragenlijsten Responstijd e-Q)
aantal keer 10000 gerappelleerd 0 1 8000 2

Paper Responstijd voor papieren vragenlijsten (letter + Q)

10000

Reminder 1

Reminder 1

8000

Number offrequentie responses

Reminder 2

Reminder 2

6000

frequentie
0 3 6 9 1 1 1 2 2 2 3 3 3 3 4 4 4 5 5 5 6 6 6 6 7 7 7 8 8 8 9 9 9 9 1 1 1 1 1 1 1 2 5 8 1 4 7 0 3 6 9 2 5 8 1 4 7 0 3 6 9 2 5 8 1 4 7 0 3 6 9 0 0 0 1 1 1 2 2 5 8 1 4 7 0

6000

4000

4000

2000

2000

0
0 3 6 9 1 1 1 2 2 2 3 3 3 3 4 4 4 5 5 5 6 6 6 6 7 7 7 8 8 8 9 9 9 9 1 1 1 1 1 1 1 2 5 8 1 4 7 0 3 6 9 2 5 8 1 4 7 0 3 6 9 2 5 8 1 4 7 0 3 6 9 0 0 0 1 1 1 2 2 5 8 1 4 7 0

2/10/2012 afgerond Days responstijd Days in dagen

responstijd afgerond in dagen

16

Getting Data for Business Statistics

Macro paradata
R-indicator to monitor fieldwork of business surveys

The representativity of the Monthly Survey for industry and retail trade by number of fieldwork days.
0,5
Retail

0,9

Industry

0,4 Maximum absolute bias

Industry

R -indicator

0,8

0,3
Retail

0,7

0,2

0,6

Retail

0,1
Industry

0,5 0 10 20 30
2/10/2012 Days

0,0
40 50 60 70

10

20

30 Days

40

50

60
17

70

Getting Data for Business Statistics

Micro paradata audit trails

800 700

Completion process e-SBS: conscientious R

session no. day no. help
600

info calculator save print send action

500

400

300 'question number'

200

100

0 0:00

0:17

0:22

0:30

1:31

1:55

2:02

2:06

2:19

2:26

2:42

2:53

2/10/2012

time (hour:min)

18

Getting Data for Business Statistics

Micro paradata: audit trails

800 700

Completion process e-SBS: quick n dirty R

session no. day no. help
600

info calculator save print send action

500

400

'question number'

300

200

100

2/10/2012

0 00:00

02:17

02:57

03:46 time (min)

03:51

04:02

04:16

19

Completion N times started

Min. 1

Max. 37 11:29:51

Average 1.7 01:07:30

Completion time 00:01:27

Used functionalities of the Questionnaire Print button Save button

Used by How many times % used of Rs mean min.-max. 43 28 1.7 2.6 1-21 1-95

2/10/2012

20

Getting Data for Business Statistics

The data collection design today

More complex than yesterday: More data sources

2/10/2012

Dependent on providers of registers Integration of sources Coordinated developments over modes Tailoring to mode Tailoring to target populations Coordination over surveys (samples and Qs)
21

Mixed-mode surveys

Tailoring to respondents

Tomorrow, even more complex

Getting Data for Business Statistics

Tomorrow
Multi-source/mixed-mode data collection
Managing integrated sets of statistics (not stove-pipes) Advanced statistical modelling and estimation Coordinated data collection designs:
- not single-purpose, but multi-purpose surveys

Advanced questionnaire design:

- images, spoken language, animations, video pictures

Methodologists: competent in all modes Process measurement and quality control:

- continuous measurement using paradata

Opening the survey process

- responsive adaptive designs Tailoring to the internal businesss processes Improved communication with businesses Insight in the internal response processes
22

Opening the businesses

2/10/2012

Getting Data for Business Statistics

Whats next?
Opening the businesses
Insight in the response processes

A Business CASM movement:

Communicative Aspects of Business Survey Methodology
Communication sciences Administrative sciences Organisational sciences Psychology (organisational, work and social, cognitive)

2/10/2012

23

Statistical picture of a country

External business factors Econ. climate Regulatory requirements Political climate

Internal business factors Policy Data Resources Market position Informant: Mandate Data knowledge Job priority Image

The survey: Topic Population and sample Sponsor / Survey organisation Resources Planning Authority/confidentiality

The survey design Questionnaire Modes of data collection Contact strategy

Motivation Black box Decision to participate

Answering behaviour

Response
In time Complete Correct

Respondent burden
De facto Perception

Register data

More than one survey

More than once In other ways: Registers EDI

NSI

2/10/2012

A business

24 NSI

Getting Data for Business Statistics

Communication model
Direct communication

Response
in time, complete, correct

Decision to participate

One coherent strategy with regard to tone-of-voice, lay-out, and compliance principles

Indirect communication

Image NSI Communication we cannot control

2/10/2012

25

References in addition to proceedings paper

Bethlehem, J., F. Cobben, and B. Schouten (2008), Indicators for the Representativity of Survey Response. Presentation at the 24th International Methodology Symposium of Statistics Canada: Data Collection: Challenges, Achievements and New Directions, 28-31 October 2008, Gatineau, Canada. De Nooij, G. (2008), Representativity of Short Term Statistics. Statistics Netherlands, The Hague. Groves, R.M. (2008), Dynamic Survey Design managed by modelled Paradata. Presentation at the 24th International Methodology Symposium of Statistics Canada: Data Collection: Challenges, Achievements and New Directions, 28-31 October 2008, Gatineau, Canada. Scheuren, F. (2001), Macro and Micro Paradata for Survey Assessment. Urban Institute: unpublished paper, Washington D.C., USA. Snijkers, G. (2007), Collecting Data for Business Statistics: Yesterday, Today, Tomorrow. Presentation at 56th Meeting of the ISI, 22-29 August 2007, Lisbon, Portugal. Snijkers, G. (2008), Getting Data for Business Statistics: A Response Model for Business Surveys. Presentation at the 4th European Conference on Quality in Official Statistics, 8-11 July 2008, Rome, Italy.
2/10/2012 26