* Secondary Data Analysis:

An Introduction
Dr Juliet Hassard
Deputy Director, Centre for Sustainable Working Life
Lecturer in Occupational Health Psychology
*What is secondary data
analysis?
*Types and sources of data
*Opportunities,
limitations, and
challenges
*Ethics
*Thinking forward: funding
and publishing.

*Overview of
presentation
*Secondary data
analysis: why and
what it is?

* Traditionally, the field of
psychology (any many of those
within it) have dismissed the
importance and value of
studies using secondary data.
* But times are changing…….
* The use of secondary data, or
existing data that are freely
available to researchers who
were not involved in the
original study, has a long and
rich tradition in the social
sciences [1].
* Sociology, economics, etc.
* Why collect new data, given
the wealth of existing data
sets that can be used to
answer important questions?
* Longitudinal & large sample
sizes.
*Why use secondary data?
* To ask and answer
important questions.
For example,
* To understand the
longitudinal nature
of relationships.
* To understand
group differences,
trends over time?
* To explore new and
emerging social
phenomena.
 * More data (and types of
data) are being collected
(and available!) then ever
before.

* There is a unique
opportunity to explore this
ever growing source(s) of
data, and to ask important
research questions.

*Why secondary data

analysis?
* Let’s get creative……..

* In small groups of 3-5. Discuss and outlines 4-5

different types of data/ types of information
that could be used to investigate an important
psychological research question.

*Types and sources of

data….
 Blogs Chat forums

Published
business
reports

Online support groups

Second life
App technology
*Where do I find * The UK Data Service
* https://www.ukdataservice.ac.uk /
data? * Census data
* International macrodata
* Longitudinal studies
* Qualitative/mixed methods
* UK surveys

* The National Data Service

* http://www.nationaldataservice.org/about
/

* Individual studies may have different

access points.
* E.g., Whitehall II Study, UCL.

* Secondary data is everywhere – some in

the public forum.
* Online support groups:
* COULSON, N.S., 2015. Exploring patient's engagement with web-based peer support for
Inflammatory Bowel Disease: forums or Facebook? Health Psychology Update. 42(2), 3-9.

* Longitudinal data (Whitehall II survey)

* Kouvonen, A., et al . (2011). Negative aspects of close relationships as a predictor of
increased body mass index and waist circumference: the Whitehall II study. American
journal of public health, 101(8), 1474-1480.

*Twitter, Instragram…..
* Whiting, R., & Pritchard, K. (2015). “Big Data? Qualitative Approaches to Digital Research",
Qualitative Research in Organizations and Management: An International Journal, Vol. 10
Iss: 3, pp.296 - 298

*Examples
*Advantages, Limitations,
& Challenges
 Low
response High attrition Access to high
rate rates quality
measures

Small sample size

Limited money
& resources
to collect
‘Traditional’
Reliance on primary data
Challenges in
convenience Psychological Research
samples

Limited scope for

extensive
comparative research
Correlation does
(across groups or
not equal causation
internationally)
 * The data has already been
collected.

* Save time – primary researcher

does not have to design study
and collect new set of data.

* The types of data that are

typically collected tend to be
higher quality than could be
obtained by individual
researchers.

* Typically longitudinal, have large

sample sizes that have been

*Advantages
obtained using elaborate sample
plans.
Ref: Trzesniewski et al., 2011
 * Learning how to work with,
manage and analyse
secondary data can provide
individual researchers with
the raw materials to make
important contributions to
the scientific literature
* … using data sets with
impressive levels of
external validity.

Ref: Trzesniewski et al., 2011

*Advantages
* Open-source approach to research
* Replicate findings using similar analyses
* Encourages careful reporting and justification of
analytical decisions.
* Allows researchers to test alternative explanations and
competing models.

* Encourages transparency, which in turns help

facilitates good science.

Ref: Trzesniewski et al., 2011 *Advantages

 *The data has already been
collected!!!
* Disadvantages
* You may not have all the
information on how or why
certain types of information
was collected.

* You may not know of any

particular problems that
occurred during data collection.

* Sometimes you are left wanting

more …..
* The temptation: a statistical fishing trip.

* Great research is driven by a good research question

that is strongly underpinned and shaped by theory.

* The purpose of analysing data is to refine the scientific

understanding of the world and to develop theories by
testing empirical hypotheses.

* “Mo Money Mo Problems” - Mo Data, Mo Temptations ?

* A note about statistical power.

Ref: Trzesniewski et al., 2011

*Disadvantages
 * Considerable time and effort:

* is invested by the researcher

to understand the nature and
structure of a data set.

* is needed by the researcher

to explain and justify the
theoretical and analytical
approached used.

* Although, I would argue there

is real advantages to the time
invested in doing this.

*Disadvantage
Ref: Trzesniewski et al., 2011
 * Measures in these datasets are often abbreviated. Often
because the projects themselves were designed to serve
multiple purposes and to support a multidisciplinary team.
* Shortened measures, mix-levels of data, and single items
measures.

* These datasets often have impressive levels of breadth

(many constructs are measured), but often with an
associated cost in terms of depth of measurement.

* Therefore, measurement issues are ~ therefore ~ one of

the major issues in secondary data analysis
* These issues often require quite a bit of conceptual
consideration & defending in the peer-review process.

Ref: Trzesniewski et al., 2011

*Disadvantage
* A good grounding in psychometrics and Classic Test
Theory.

* You need to carefully consider and evaluate the trade-offs in

reliability and validity.

* You need to defend your position when writing up.

* You need to understand how measurement issues frame your

findings; and, in turn, your interpretation of your findings.
Ref: Trzesniewski et al., 2011

*Challenges
* Creating and managing data files
* Data inventory
* Research journal

* Approach to missing data and

data screening procedures

* Use of and/or development of

constructs
* Use of proxy variables
* Development & testing of
composite measures
* Single item measures
*Practical &
* Accounting for the data
structure in your analysis Methodological
Challenges
*Case study: An
Example
 *
MODELLING GENDER-RELATED DIVERSITY IN PSYCHOSOCIAL PROCESSES AND WORK-
RELATED WELLBEING: PATHWAYS AND MECHANISMS

* The aim of the doctoral thesis was to develop and

test a theoretical model seeking to describe the
aetiological role of psychosocial processes, in and
out of the workplace, in predicting gender-related
diversity issues in men’s and women’s health at a
structural/population level.

* An iterative multi-stage methodology was utilised

to develop and test the proposed theoretical
model.
 • Literature review – Theoretical framework
Stage one

• Identification of suitable source of data

Stage two

• Data review (data inventory)

• Measurement development and testing
Stage three • Data cleaning

*Case Study:
Methodology
* European Working Conditions
Survey

* Pan-European cross sectional survey

of working conditions, worker’s
health and safety, and living
conditions (n = over 40, 000
workers)
* Now on the 6th wave of data
collection.
* The survey as evolved over time
asking more questions.
* Survey items are informed and
based on contemporary theory
* The measures used are not always
based on a validated psychometric
* Case study
measures
* Single items vs. composite
measures?
* The vast majority of latent conceptual
constructs are complex and multifaceted in
nature.

* Consequently, the use of a single item as a

theoretical concept may not yield an accurate,
comprehensive, and reliable measurement of
the given construct of interest.

*Case study: Single

item measures
* The guiding premise by many in the scientific
community is that multiple responses reflect the
“true” response more accurately than does a single
response.

* Imprecision in measurement is one of thekey causes

(although not the sole cause) of measurement error.

* Measurement error creates ‘noise’ to the observed

variables.

*Case study:
Measurement error
* Inaccurate and unreliable measurement of a
concept results in key concerns regarding the
overall validity and reliability of the hypotheses
tested using this (or these) given measurement(s).

* It is generally agreed/ suggested that research

findings that are valid, reliable and generalizable,
are built on a solid foundation of accurate and
consistent measurement.

* Case study: Implications of

poor measurement
* The primary objective of creating a series of
summated (or composite) scales is to avoid the
exclusive use of, or dependence on, single item
constructs where possible.

* The use of several variables as indicators

provides an opportunity to represent differing
facets of a given concept, with the aim of
yielding a more well-rounded perspective and,
arguably, a better measurement of the given
concept

*Composite measures
*Thinking about ethics
 *Researchers need to ask: how was consent obtained in the
original study? Where sensitive data is involved, we cannot/
should not assume informed consent.

*Given that it is usually not feasible to seek additional

consent, a professional judgement may have to be made
about whether the use of secondary data violates the
contract made between subjects and the primary researchers.

*Growing interest in secondary data make it imperative that

researchers in general now consider obtaining consent, which
covers the possibility of secondary analysis as well as the
research in hand.
* This is consistent with professional guidelines on ethical
practice

*A note about ethics

Heaton, J (1998). Secondary analysis of qualitative
data. Social Research Update (issue 22). See:
http://sru.soc.surrey.ac.uk/SRU22.html
* Can you publish secondary data analysis – yes!

* Never forget: the central role of theory.

* Be detail orientated!

* Justifying your research question is important, but you also need to be

prepared to justify and outline the logic of your analysis framework
and approach.

* Understand and reflect on how the research design or any experienced

methodological issues of your secondary data may impact or frame the
interpretation of your results.

*Some thoughts on
writing up
* Secondary data analysis is an important and
useful research methodology.

* There are many benefits and strengths to using

secondary sources of data.

* But there also important pragmatic and

methodological challenges that face
researchers.

*Conclusion
* Trzesniewski, K. H., Donnellan, M., & Lucas, R. E. (2011). Secondary
data analysis: An introduction for psychologists. American
Psychological Association.

* Vartanian, T. P. (2010). Secondary data analysis. Oxford University

Press.

* Heaton, J. (2008). Secondary analysis of qualitative data: An overview.

Historical Social Research/Historische Sozialforschung, 33-45.

* Hinds, P. S., Vogel, R. J., & Clarke-Steffen, L. (1997). The possibilities

and pitfalls of doing a secondary analysis of a qualitative data set.
Qualitative Health Research, 7(3), 408-424.

*Suggested reading
* Thank you for
listening
j.hassard@bbk.ac.uk