LDC e-learning guides

Evaluating E-Learning

Ref: Eval-EL-G3

Undertaking a needs analysis for e-learning

What is a needs analysis? ................................................... 1

How it aids design and planning ......................................... 2
Addressing stakeholder needs ............................................ 4
Involving students .............................................................. 4
What should be considered ................................................. 3
Using models and templates ............................................... 4
SWOT ..................................................................................................... 4
Profiling ................................................................................................... 5
Curriculum design ..................................................................................... 6

Matching e-learning methods to needs ............................... 6

Author..................................................................................................... 8

References and further resources ....................................... 8

What is a needs analysis?

Undertaking a needs analysis for e-learning

May-04

Needs analysis is an element of designing (or reviewing) a curriculum. Its purpose is

to establish key learning outcomes and requirements in the design and delivery of a
course or learning activity. The needs relate to the characteristics, concerns and
potential constraints of the students (or any other relevant stakeholders). The analysis
seeks to match possible or proposed techniques and materials to these needs and thus
identify whether the design is appropriate to the intended goals.
Good course design should separate ends from means. We are constantly making the
mistake of specifying the means of doing something rather than the results we want.
This can only limit our ability to find better solutions to real problems. (Gilb, 1988)
In most cases, reviewing a course and responding to current need is perhaps
something done intuitively and without formal procedures. However, there is
increasing pressure to update curriculum purposes and methods in response to
changing government requirements (such as accessibility, employability and
information and IT skills agendas). Developing a new course or changing an existing
teaching approach is likely to feel daunting, time-consuming and risky, especially
when technology is involved. These risks and concerns are likely to be significantly
diminished if a more explicit approach is taken to evaluating needs. There is certainly
the usual need to justify limited time available and to be aware of likely technical
requirements.

How it aids design and planning

A needs analysis is an effective means of identifying objectives and requirements for
e-learning development. Understanding students needs (not to mention your own!) is
crucial to the successful design or redesign of any course or learning activity. Needs
analysis for learning is also one of the most difficult things to do well. The use of
templates or models, as well as availability of training and support in e-learning, can
reduce the risk and isolation of trying out new methods.
If needs or requirements are unclear the specification for what you are developing
will be wrong. If the specification is wrong then the design will be wrong. If the design
is wrong the students will be dissatisfied or not achieve what you/the course
intends.
These four real world scenarios have common features aside from ambiguity.

The University is concerned about equal opportunities in the curriculum. The

present induction course is out of date. As a result, a working group is
established to rejig the existing awareness programme for departments.

At a departmental meeting, the Chair says: We have all these new skills
policies. We need to make sure all curricula are explicit about what employable
skills are being developed by our students. We need all tutors to review their
courses in terms of using e-learning to support skills development.

A lecturer has just been given a set of multimedia web materials that support
students in critically evaluating X (problem/issue/topic). S(he) is now tasked
to ensure they know how to use it and what to do with it.

A course team are keen to use e-learning to encourage collaborative learning.

They are planning to use online discussion tools to support an existing course
taught with face-to-face lectures and web resources.

Undertaking a needs analysis for e-learning

May-04

A needs analysis will assist in all of these scenarios in terms of identifying/clarifying

staff or students needs and producing clear and measurable outcomes as indicators of
success of the (e-learning) development.
Further aims of a needs analysis are to support the selection of approaches that
achieve one or more of the following:

are likely to save time or costs

are valuable and viable
are scalable and sustainable.

It is fairly well accepted that e-learning developments that are valuable and
sustainable in the longer term are those based on the use of small-scale, incremental,
non-revolutionary technologies, i.e. mainstream worldware tools (Ehrmann, 2000).
These require far lower investment in terms of cost, maintenance, updating and skills.
In todays climate, the Web is the ultimate worldware tool; this includes web-based
communication tools, but also analysis type applications, such as spreadsheets, maths
tools, design software etc.). The challenge to the lecturer then is to package these
basic tools in pedagogically viable ways. This is where needs analysis can assist in
providing diagnostic evaluation that is, scoping out the objectives of what your
development seeks to achieve against the requirements of the intended end-users.

What should be considered

A needs analysis is worth spending some time on so you go about things in a
systematic manner. Evaluation should ideally be planned at the outset of any new
development or modification. Your initial plan is likely to include aims, questions,
tasks, stakeholders, timescales and instruments/methods. It should define that which
you are trying to investigate but also how you are going to go about it (Crompton,
1997).
Generally speaking, the areas to consider in embedding e-learning effectively into a
course include issues around:
1. Learning- is the pedagogy appropriate?
2. Infrastructure will the environment support my needs?
3. Technology is the technology appropriate and can I/students use it?
Identifying and analysing preconceptions and assumptions prior to use can ensure
potential barriers to access or effectiveness are accounted for appropriately in
development design and planning
A preliminary review is likely to include:

Student and curriculum needs analysis tasks

Constraints, resources and costs

Identifying ones own professional development needs

Identifying evaluation criteria (indicators of success)

Creating a statement of purpose (for the development project)

Undertaking a needs analysis for e-learning

May-04

Addressing stakeholder needs

Needs analysis is closely linked with evaluation and dissemination. Firstly, it is the
crucial diagnostic part of an effective evaluation (see LDC e-learning guide
Evaluating e-learning developments). The purpose of evaluation is to offer a means
to investigate, provide evidence, learn, share and make judgements about what we do
and how we do it. Secondly, there are inevitably links with feedback gathered from
dissemination activities, as you may engage with stakeholders who can inform the
choices you make in your development.
A needs analysis is the starting point for defining the criteria against which
judgements about success can be made. It will include a close scrutiny of the
pedagogical rationale and outcomes of the curriculum and assessment design, which
the planned e-learning (activity or environment) is intending to support or enhance.
Some of the areas your needs analysis should address may be determined by your
stakeholders (those who are influenced or benefited by the development). Primarily,
this is most likely to be your students (whose learning you are aiming to enhance), it
may be yourself in terms of developing your own skills. However, there may well be
other, perhaps departmental, concerns or questions which you might choose to
include for others treading a similar path.

Involving students
Involving students in the needs analysis process helps you to engage with them as the
main beneficiaries (or sufferers!) of any new e-learning approaches you put in place.
It can encourage more active participation in the development process. You might
consider holding an initial focus group with students explaining your aims and the elearning being developed and ask for their ideas and feedback as things progress. The
students can also propose areas for investigation and can give feedback on the
effectiveness of your evaluation questions.
Many techniques are available for eliciting student (user) needs. One problem is that
students often do not know or cannot articulate what they want in the course context
For example, new students who have not studied a subject beforehand may not have
the necessary language/terminology. They may lack the knowledge that comes from
experience of using technology: selecting those that have experience of e-learning
activities. Often students who volunteer for focus groups or interviews or return
questionnaires are the technophiles, the technophobes not wishing to publicise their
lack of skills. This can skew your analysis considerably. Selection of students to
involve in a needs analysis therefore requires careful thought, to ensure you obtain
responses that are reasonably representative of the whole student group. You may
have to use elicitation techniques that build knowledge as well as keep a focus on the
users own needs.

Using models and templates

SWOT
SWOT stands for Strengths Weaknesses Opportunities Threats. It is a standard
brainstorming and communication technique used to identify issues associated with
change. Before you (as tutor) consider what you think e-learning is and its pros and
cons, a useful exercise is to draw up a grid and quickly fill in what your general
attitudes are. For a template and guidance on this, see appendix 1.

Undertaking a needs analysis for e-learning

May-04

An example of this exercise given in the table below from a session with a group of
academic tutors analysing the broad benefits and constraints of e-learning.
Strengths

Weaknesses

Possibility of more interaction than

with a lecture
Visualisation (graphics, animation)
better
'Individual' attention is possible
(although not personalised)
Self-paced (but limited if part of a
conventional course)
Allows for different level of user
Consistency
Cheap (but only if it has a long
technology/content 'lifetime')
Frees staff for more effective faceto-face work than mass lecturing
Consistency

Opportunities

Lack of ownership (how do you

underline and scribble notes in the
margin?)
Students need to be quite well
motivated (better for PG courses?)
'Self-management' culture difficult for
some students
Needs to be balanced with traditional
methods - some students don't like
interacting with machines
Training needed for to use software
effectively
Loss of face-to-face richness
Lack of interactivity (compared with a
tutorial or lab)
Expensive to set up
Can become over-focused on
technology at the expense of content
and method
Threats

More flexible access to learning

Can reach more students over a
range of times and locations
Can deal with more students
Better quality?
May enable more choice of content
May be more democratic/egalitarian
Education can be tailored (Just in
time/Just enough/Just for you)

Commercial bodies my will use it to

compete with conventional HE
Government (mistakenly?) believes it
is cheap
Job losses?
Learning 'facts' may overshadow
learning 'experience'
May turn HE into a 'learning
supermarket'
Less opportunity to ask the lecturer
questions
Move away from traditional university
values

Profiling
Simple documentation templates can be created for recording and developing user
needs statements from an interview or focus group. Questions and discussion might
be used to gather information on the student group:
Students availability or preference of study environments
Student characteristics: entry qualifications, employment aspirations,
language, disability, etc.
Current student problems or concerns
Refining an existing methods or provision
Establishing what students see as appropriate or helpful activities
Recording such information systematically over time helps develop a culture of
considering student needs and experiences as well as gauging successes of a new
course or method.
Undertaking a needs analysis for e-learning

May-04

Curriculum design
A needs analysis for a new method or course is likely to include a review of overall
teaching strategies. E-learning facilitates a whole range of teaching and learning
activities see LDC e-guide E-learning solutions for teaching and learning. However,
a learning activity does not take place in isolation to the teaching and study
environments, assessment tasks, tools used and so forth. It is therefore important to
ensure that all components of the curriculum, including any uses of e-learning
methods or materials, are properly integrated and the purpose of a particular
component is then clear to the student.
Pedagogical models are useful for making explicit the intentions of the e-learning
approach. The needs of the tutors and students will be different in each case.
Looking at needs across different types of development models, Robin Mason
(1998) contends that:
Current approaches to teaching and learning in higher education are
dominated by the following: the importance of interactivity in the learning
process, the changing role of the teacher from sage to guide, the need for
knowledge management skills and for team working abilities, and the
move towards resource-based rather than packaged learning.
She suggests three development models:
1. CONTENT SUPPORT MODEL
Course content and tutorial support are dealt with separately, particularly used where
content does not change significantly or where courses are tutored by external staff.
Collaborative activity (peer commenting, online assessment, computer conferencing)
amongst students is rudimentary, in most cases less than 20% study time, and added
onto the course as supplementary rather than core. With increasing use of the web for
delivering content, there is more scope to extend the balance of content to
collaborative activity.
2. WRAP-AROUND MODEL
The course content consists of tailor made materials (study guide, activities and
discussion) wrapped around existing materials (textbooks, CD-ROM resources or
tutorials) and representing around 50% study time. The remaining 50% is comprised
of online interactions and discussions, including real time online events and screen
sharing with increasing audio/video components. The tutor role is more extensive as
less of the course is pre-determined and students take more responsibility for their
learning.
3. INTEGRATED MODEL
The third model is at the opposite end of the spectrum from the first. The heart of the
course involves collaborative activities, learning resources and joint assignments.
These take place online through discussion, accessing and processing information and
carrying out tasks. The course contents are fluid and dynamic as they are largely
determined by the individual and group activity. In a sense, the integrated model
dissolves the distinction between content and support, and is dependent on the
creation of a learning community.

Matching e-learning methods to needs

Learning is not a spectator sport. Students do not learn much just sitting in classes
listening to teachers, memorizing prepackaged assignments, and spitting out answers.
They must talk about what they are learning, write reflectively about it, relate it to
past experiences, and apply it to their daily lives. They must make what they learn
part of themselves. (Ehrmann, 1995)
Undertaking a needs analysis for e-learning

May-04

The range of technologies that encourage active learning is staggering. Many fall into
one of three categories: tools and resources for learning by doing, time-delayed
exchange, and real-time conversation.
Building on existing course models, it can be useful to map tools and technologies
onto learning activities and to consider how these enhance or enrich what you already
do. A simple list to use might look like this:

Lecture presentation
Knowledge dissemination
Communication with and between students
Assessment for feedback and monitoring
Labs and tutorial activities
Course management

Pedagogically, you might consider what learning theories are being supported (see EGuide Pedagogies for E-Learning). For instance:

Instructivist
Constructivist
Situated learning

Or more specifically, for example:

Knowledge of facts and figures

Problem-based learning
Work-based learning
Practice-based learning
Critical thinking skills
Analytical skills
Creativity

The appropriateness of tools that are orientated towards content, communication or

collaboration then (hopefully) starts to become more obvious.

Reviewing your course

An initial course design review is likely to include the following questions:
1. What is the course or module?
Identify/clarify the aims, objectives, format, student LDCabilities to be
developed, core content etc.
2. What are the student group characteristics?
Identify/clarify student numbers, backgrounds, special needs or languages, IT
literacy, access to computer/network, UG/PG, level etc.
3. What is the development project intended to achieve?
Identify/clarify the main aims and objectives for what you wish to develop,
intended benefits and outcomes, sources of background literature.
4. What are the implications for the technologies and tools selected?

Undertaking a needs analysis for e-learning

May-04

If possible, identify existing examples of use of the technologies you are

proposing to integrate, particularly those available locally, such as application
type, access/availability of licences, robustness, network requirements,
usability issues, any costs of developing or extending software programs.
5. What skills do I need to develop to be an effective teacher using these tools
and online environments?

Author
Dr Jay Dempster, Learning Development Centre
Tel: 024 76524670 Email: jay.dempster@warwick.ac.uk

References
Chickering, A. and Ehrmann, S. (1995) Implementing the Seven Principles:
Technology as Levers
Compton, Philip (1997) Evaluation: A practical guide to methods, Philip, in LTDI
Implementing Learning Technology: http://www.icbl.hw.ac.uk/ltdi/implementingit/eval.htm
Ehrmann, Stephen C., (2000) Technology and Revolution in Education: Ending the
Cycle of Failure, Liberal Education, Fall, pp. 40-49. Available on the web:
http://www.tltgroup.org/resources/V_Cycle_of_Failure.html
Gilb, T. and Finzi, S. (1988) Principles of Software Engineering, Addison Wesley.
Mason, R. (1998) Models for online courses. ALT Magazine online at:
http://www.aln.org/publications/magazine/v2n2/mason.asp

Further resources
Pedagogies for E-Learning, LDC e-learning guide:
http://www.warwick.ac.uk/go/LDC/resources/eguides/eskills/pedagogies/
E-Tutoring: Teaching, Supporting, Managing and Assessing Students Online, LDC elearning guide:
http://www.warwick.ac.uk/go/LDC/resources/eguides/eskills/etutoring/
Biggs, J. (2002) Aligning the curriculum to promote good learning. LTSN Generic
Centre paper from Constructive Alignment in Action Imaginative Curriculum
Symposium Nov 2002. Available on the web at: http://www.ltsn.ac.uk/
application.asp?app=resources.asp&process=full_record&section=generic&id=156
Collis, B. (1996). Tele-learning in a Digital World. Thomson Computer Press, London.
Petersen, R. (1992) Training needs analysis in the workplace, London: Kogan Page.
Stoner, Greg (1996) A conceptual framework for the integration of learning
technology, chapter 3 in Implementing Learning Technology, LTDI publication.
http://www.icbl.hw.ac.uk/ltdi/implementing-it/implt.pdf, pp. 6-8.

Undertaking a needs analysis for e-learning

May-04

Appendix 1: SWOT Analysis of e-Learning Technologies

Strengths:

What does the technology do well?

What could you use it to do?
What relevant support, resources, experience, training do you have access to?
What do students see as the strengths of the technology? What do they like
about it?

Consider this from your own point of view and from the point of view of the people
you deal with. Be realistic. If you are having any difficulty with this, try writing down a
list of the features of the technology. Some of these will hopefully be strengths!
In looking at the strengths, think about them in relation to the alternative methods for example, how does an online discussion compare to a seminar for achieving
existing learning goals and possible new learning goals?
Weaknesses:

What
What
What
What

does the technology do badly?

resources, support etc do you lack?
should you not use it for?
do students dislike about it?

Again, consider this from an internal and external basis: Do other people seem to
perceive weaknesses that you do not see? Is the traditional method actually better at
delivering the learning goals? It is best to be realistic now, and face any unpleasant
truths as soon as possible.
Opportunities:

Where are the good opportunities facing you?

What are the interesting trends you are aware of?
Useful opportunities can come from such things as:
o Changes in technology provision / support
o Changes in University strategy
o Changes in social patterns, population profiles, lifestyle changes, etc.
Ubiquitous computing, Lifelong learning.

A useful approach to looking at opportunities is to look at the strengths of the

technologies and ask yourself whether these open up any opportunities.
Alternatively, look at the weaknesses of the technologies and ask yourself whether
you could open up opportunities by eliminating them.
Threats:

What obstacles do you face?

Do you have support of your department and support services in using this
approach?
Are you likely to be able to invest sufficient time in the project? What might
prevent this?
Could any of the weaknesses seriously threaten the success of your project?
How much does the success of the project depend on you personally or on
other individuals?
Carrying out this analysis will often be illuminating - both in terms of pointing out
what needs to be done, and in putting problems into perspective.

Undertaking a needs analysis for e-learning

May-04