Evaluating the impact of ICT

Prof. Angela McFarlane

University of Bristol
Impact 2

2 Preliminary studies
Main team at Nottingham, MMU and OU
60 schools
Teacher researchers
Pupil researchers
Ends July 2002 funded by DfEE
Impact 2 - Strands
Strand 1: to develop and apply appropriate
methodologies for evaluating the use of ICT in
school and to analyse the relation between effective
implementation of ICT and standards of
performance in national tests.

Strand 2: to develop and apply a variety of methods to

establish how pupils use ICT out of school and what
is gained from such use.

Strand 3: to explore the nature of teaching and

learning involving ICT in various settings, with a
central focus on the perceptions of pupils, teachers,
parents and managers.
Impact 2 - Questions
⌧(1) What is the involvement of pupils with computers and
the Internet at home and in school?
⌧(2) Does curriculum centred usage have a measurable
effect on performance and attitude ?
⌧(3) Are these effects confined to usage in school?
⌧(4) Are all kinds of computer usage equally productive of
learning?
⌧(5) If ICT based learning involves interactions between
home and school, what are the attendant problems
and how can these be met?
Impact 2 – Answers?
75% primary pupils, 90+% secondary pupils have a computer
at home
Use in school subjects – never or hardly ever
5 from 13 subjects show a small positive correlation between
ICT use and added value of attainment
Use at home seems to correlate to improved attainment in
school
The data on types of ICT usage was inconclusive
Problems of home/school use subject of a separate study
 Impact 2 – Answers?
⌧ • Differences in attainment associated with strength of ICT involvement
were never large, but were nonetheless clearly present in more than a third of all
comparisons
⌧ • In none of the 13 comparisons was there a statistically significant advantage
to groups with lower ICT involvement
⌧ • At Key Stage 2, there was a highly significant positive association between
ICT and SATs for English
⌧ • At that key stage there were similar trends for Maths and Science, but these
were less clear-cut
⌧ • At Key Stage 3 there were no clear-cut associations between ICT and SATs
results, but there were some indications of a positive association in Science
⌧ • At Key Stage 4, there was a highly significant positive association between
ICT and GCSE for Science and for Design Technology
⌧ • Also at Key Stage 4’ there were strong indications of a positive association in
Modern Foreign Languages, and somewhat weaker in Geography.
Impact 2 – Answers?

There is no consistent relation between the average amount of

ICT reported for any subject at a given key stage and its
apparent effectiveness in raising standards.

It therefore seems likely that the type of involvement is all-

important.
ICT can make a contribution to
educational outcomes

•for some learners

•under certain circumstances.

Perspectives on ICT

ICT as a set of skills/competences

ICT as a vehicle/tool set etc to do what
we have always done - better/cheaper
ICT as an agent of change which impacts
in a revolutionary way
 Aspects of ICT that can contribute to learning

Feedback

Representation of dynamic processes

The ability to edit

Multiple representations of knowledge

Shared records of learning

All depend on the prevailing learning culture

What should we look for
in terms of impact of
ICT?
Classification by outcome:

• The teaching of specified skills or discrete content e.g. spelling, the use of a
database, the three times table.

• The exposition of personal knowledge e.g. writing an essay on coffee

production, or authoring a multimedia presentation on drugs awareness.

• The building of personal knowledge through the identification, collection

and relation of relevant information sources e.g. collecting information on
coffee production or drug abuse from a variety of sources.

• Communication within an electronic community

 Traditionally
Traditionally
defined
defined ICT
curriculum and
skills
learning culture

Learner activity,
using new
technology and
ICT skills

Direct Impact Model

Improved
knowledge and
understanding

Improved
attainment
Common gaps in the data set

Number of pupils
Number of schools
Age of pupils
Number of LEAs

Computer/pupil ratio
Types of ICT used
Context used
Methodology
Date study began
Duration
Funding source
Self-directed learning Overlap Institutional learning

Traditionally
Traditionally Traditionally
Traditionally defined
defined
Learner-defined
Learner-defined defined
definedICT
ICT curriculum
curriculum and
and learning
learning
curriculum
curriculum and
and culture
culture skills
skills culture
culture

Learner’s
Learner’s personal
personal
Home-based Teacher
Teacher//curriculum-
curriculum-
representation
representation ofof the
the Home-based
task specified
specifiedtasks
tasksfor
forthe
the
task, and available
task, and available task
learner
learner
resources
resources

Socially
Potential
Potential benefits
benefits for
the
the learner:
learner:
for
Contextualised
Learner
Learneractivity,
activity,using
using
knowledge
knowledge and
(beyond
(beyond those
those
and skills
skills ICT
ICTskills
skills Impact
validated
validated or
externally)
or approved
externally)
approved
Model
autonomy
autonomy and
and
confidence
confidence in
in learning
learning Improved
Improvedrelated
related
extended knowledge
knowledgeand
and
extended awareness
awareness understanding
of understanding
of the
the social
social benefits
benefits of
of
the
the ICT
ICT

Improved
Improved knowledge,
knowledge,
understanding
understanding andand Improved
Improvedattainment
attainment
skills,
skills, of
of personal
personal and
and
social
social benefit
benefit
Two fundamental questions for the next five years -

What should be recognised as

educational attainment?

What are the implications of the wider

context of school based education?
Evaluating the impact of ICT

Prof. Angela McFarlane

University of Bristol