Exploring Models

and Partnerships for

e-learning in SMEs
Disclaimer

This publication has been produced with the support of the European Commission.

Whilst every attempt has been made to ensure that the information contained within this
booklet are correct, neither the authors, the European Commission nor the University of
Stirling will be liable for any errors or omissions contained within it.

The contents of this publication do not necessarily reflect the position of the European
Commission or the University of Stirling, nor do they involve any responsibility on their
part.

2
Exploring Models and Partnerships for e-learning
in SMEs

3
4
Table of Contents
Acknowledgements

i. Acknowledgemnts 1
ii. Foreward 2
iii. Overview 2
1. Developing new pedagogies and e-learning in SMEs.
Graham Attwell 1
2. Epistemic Activity, Organisations and Learning: towards a
framework for analysing the use of e-resources in Small and
Medium Enterprises.
David Guile 1
3. Localization of software and learning material for SMEs: how is it possible?
Bernard Blandin 1
4. E-learning Content and Software Localization.
George Bekiardis 1
5. Prospects for the development of software that truly supports
collaboration and learning in SMEs.
Graham Attwell and Mike Malloch 1
6. A framework for the evaluation of e-learning.
Jenny Hughes and Graham Attwell 1
7. Practical evaluation interventions in understanding informal learning within SMEs.
David Slater 1
8. Evaluating the effectiveness of e-learning strategies for SMEs.
Eduardo Figueira 1
9. E-learning challenges in Austrian SMEs.
Klaus Reich and Freidrich Scheuermann 1
10. Assessing the application of online learning in a work-based setting.
John Munro and Philip Crompton 1
11. A holistic vision of the future of e-learning.
Kees Schuur 1

This publication is the result of two seminars held in autumn 2002 and Spring 2003. The
seminars were supported by a grant from the European Commission DG Education and
Culture and were coordinated by KnowNet, a small research and development company
based in Wales. The main theme for the seminars was "Exploring Models and

5
Partnerships for e-learning in SMEs." There was a particular focus on the issues of
pedagogies for e-learning and evaluating e-learning.

Around 14 people attended each seminar drawn from ten different European countries
and including researchers, trainers and e-learning providers.

The seminars were intended to develop a discourse for and between research, policy and
practice in this field. We believe this aim has been achieved. The next tasks will be to
both deepen the research approach and to initiate pilot programmes to support SMEs in
developing e-learning as part of a learning culture.

May I thank all those who took part and contributed to the seminars and helped shape this
publication. In particular I would like to thank Philip Crompton from the University of
Stirling for his support and Claire Middlleton from Knownet who undertook cheerfully
the not inconsiderable task of administering the project.

6
Foreward

Graham Attwell, Knownet (UK) 
Professor Duncan Timms, University of Stirling (UK)

Digital technologies have been a major driving force behind the profound changes in
work organisation, production and society over the last twenty years. These changes have
led to what social scientists characterise as the “knowledge-based economy”, “the
knowledge society” or the “information society”, forms in which the knowledge of
individuals and organisations is critical to innovation and to economic and social
development. Whilst in previous societal forms initial education and training were seen
as providing the basic skills and knowledge required for work and for participation within
society, the new social and work forms require continuous updating of personal and
collective skills and knowledge – or lifelong learning (Attwell, forthcoming). The
knowledge society is also the learning society.
Lundvall (1996) argues that the ‘learning economy’ is a more appropriate term than the
‘knowledge-based economy’ in articulating today’s agenda where specialised and
codified knowledge has a very short life-span. It is the capability to learn how to create
new knowledge and adapt to changing conditions that will increasingly determine the
performance of individuals, firms, regions and countries (see Lundvall 1996). A
continuous process of re-learning and extending knowledge is an essential pre-requisite
for membership in the knowledge society.
The need for an extension of life-long learning, stimulated by the changes in economy
and society, has occurred at the same time that developments in communications and
information technologies offer the potential for the need to be met. Just as digital
technologies have been a key force in driving economic and social demand for new
knowledge, so they have also been hailed as the key for developing a lifelong learning
infrastructure. E-learning, the application of Communication and Information
Technologies to curriculum and pedagogy, has been seen as providing universal access to
information and as providing a new flexible and ubiquitous learning environment open to
all regardless of the constraints of time and place.
The promise of e-learning is a long time in actualization. Despite spawning a whole new
industry, not to say numerous government and European sponsored initiatives and
programmes, progress to-date has been less than convincing. The development of e-
learning has been dominated by the metaphors of the virtual classroom and the virtual
university, an over-obsession with technologies and a focus on distance applications of
existing learning opportunities, rather than the diffusion of learning in wider societal
activities and forms. Some have gone so far as to claim that the term e-learning has
become devalued to the extent where it might be more properly seen as a marketing word
rather than a description of pedagogic and learning practices (Attwell, forthcoming).

Most of the attention paid to e-learning has been concerned with its application to the
formal education system of schools and universities. There has been very limited

7
attention to vocational and occupational learning and the development of e-learning
environments in less formal learning contexts. Billet (2001) points out:

“For many workers, perhaps most, the workplace represents the only or most
viable location to initially learn and or/develop their vocational practice.”

Billett suggests that informal learning is the primary form of vocational learning – both
initial and ongoing – for most people in the workforce today. Therefore, learning from an
economic, human and social point of view has to be embedded in the fabric of all work
organisations. If e-learning is to make a contribution to changing the learning paradigm,
e-learning must be embedded in the work organisation. E-learning has until now a limited
application in small enterprises – and has tended to focus on providing networked access
to virtual classroom type environments or to distance learning supported by computer-
based materials. The present volume is an attempt to redress the lack of attention that has
so far been paid to the implications of e-learning for workers in small and medium-sized
enterprises.

8
Overview

The focus of the present volume is on the implications of Information and

Communications Technologies (ICT) for learning in European SMEs. The papers were
produced as part of a project, funded by the European Commission, DG Education and
Culture, and co-ordinated by Knownet, a small north Wales based research and
development company, with support from the University of Stirling in Scotland. The
main tool used to gather the raw material for the project was two seminars, drawing
together a range of experts from across the EU. The seminars were intended to build on
the experience and outcomes of existing European research in the field in order to focus
on the issues for developing the use of ICT for learning in SMEs and produce policy
recommendations for the European Commission and Member States.
The overall title for the seminars was Exploring Models and Partnerships for e-learning
in SMEs. Whilst the themes for the seminars were broad, the seminars focused more
tightly on issue of pedagogies for learning in SMEs and on efforts to evaluate learning
using ICT in SMEs. These were seen as critical to developing the theory and practice of
using ICT for learning in SMEs and as a subject for policy advice and development.
The first seminar was held in Stirling in November 2002 and the second in Brussels in
February 2003. Invitations to the seminars reflected two primary concerns. The first was
to ensure a dialogue between participants in different countries in Europe, in recognition
of different European traditions, cultures and economies affecting the development,
organisation and operation of small and medium-sized enterprises and the differing
patterns of technology diffusion and use in different countries in Europe. The second was
the desire to encourage dialogue between different disciplines, including researchers from
sociology, computer science, communications theory and education along with specialists
in pedagogy and evaluation. The aim of the seminars was not to solve problems as such,
but to begin to map the range of research and policy issues and to lay the basis for a
longer going research and development effort relating to the effective use of e-learning in
the small and medium-sized workplace. The papers included in this volume reflect both
the diversity of background of the participants and the aim to scope the field.

The first paper, ‘Developing new pedagogies and e-learning in SMEs’ by Graham
Attwell provides a general account of the background, discourse and conclusions to the
working sessions on pedagogy held at the two seminars.
The first seminar looked at the main issues involved in developing pedagogies for the use
of information and communication technologies in SMEs. These include definitions of
both e-learning and SMEs, managers’ attitudes to learning, the supposed cost benefits
which might accrue from e-learning and issues around standards and technology and
content.
Towards the end of the Stirling group discussions, participants started looking at the
problem of forecasting future developments and what the implications might be for
SMEs. They agreed that the real challenges were to stop thinking within existing
paradigms of learning and e-learning and to develop new ones. This raised a number of
issues.
1. One was the nature of informal (or tacit) knowledge as opposed to formal
knowledge.

9
 2. The second issue was the tension between individual learning, which some
commentators believe tends to be encouraged by e-learning, and the social
nature of adult learning as reflected in social constructivist theory.
3. The third issue followed on from this, stressing the need to ensure that e-
learning can support the sharing of knowledge and the development of new
knowledge which are vital for SMES. If e-learning and e-resources are to be a
medium for doing this, there is a need to look at the social processes involved in
online interaction and to develop new processes and solutions which can
support collaboration and co-operation.
4. The fourth issue, arguably more fundamental than the others, was the
question of what problem is e-learning trying to solve. It is difficult to evaluate
the benefit of e-learning to SMEs if it is unclear what problem it is trying to
solve and whether technical developments are going to radically change its
characteristics.
The second seminar concentrated on recommendations for policy and an attempt to
identify future research needs. The aim was to transcend the gap between research, policy
and practice.
Participants identified a wide range of potential policy issues. Three main themes were
identified as warranting further research:
1. The first was to research how SME managers and SME employees are using
ICT within their everyday work and to use this as a basis for developing
environments and opportunities for e-learning.
2. The second was to recognise an ongoing paradigm shift from e-learning to
knowledge management and to initiate research on knowledge management
including ‘How does it happen?’ - ‘What technologies support it?’ - and ‘Chaos
management, complexity management, and the management of uncertainty”.
3. The final need was for more research into social networks in and around
SMEs and their impact on knowledge sharing, development and learning.

The second paper, by David Guile, is entitled ‘Epistemic Activity, Organisations and
Learning: towards a framework for analysing the use of e-resources in Small and
Medium Enterprises’. Guile says that discussions in EU policy about the ‘knowledge
economy’ have tended to emphasise the contribution the technological output of
knowledge makes to economic development and the use of ‘e-learning’ to widen access
to education and training so that more people acquire the qualifications needed for
‘knowledge work’. By focusing on the increase in knowledge-based products and
services and on the demand for qualifications, policymakers and educational researchers
have tended to overlook issues relating to the concept of the knowledge economy and its
relationship to learning and the use of e-resources.
The first issue is that the knowledge economy does not just hinge on the increased
presence of technological and informational products of knowledge and on the volume of
qualifications. It also means that ‘knowledge cultures’ (Knorr Cetina, 1997) have spilled
and woven their tissue into the fabric of society and workplaces. A knowledge economy
is not just an economy of more experts, more technological gadgets, more specialist
interpretations, any more than lifelong learning is primarily concerned with the
accumulation of more qualifications.
The second issue is that although the level of educational qualifications workers hold is
an important issue, qualifications are not the determinants of ‘knowledge work’ (Guile,
2001). Knowledge work presupposes distinctive forms of social organisation and social

10
practices that enable people to learn to produce, circulate and use knowledge in
workplaces. Knowledge work also presuppose constant renewal and transformation,
demanding an individual and collective capacity for learning.
The aim of the paper is to provide a framework for analysing the symbiotic relationship
between knowledge cultures, learning and e-resources. In order to do this, the paper
introduces the new concept of ‘epistemic activity’. The concept refers to participation in
social practices, including the use of their associated artefacts, which facilitate the
production and application of new knowledge. The paper starts by identifying the
premises that inform the concept of epistemic activity; it then proceeds to use the concept
to analyse the symbiotic relationship between knowledge cultures and the production of
new knowledge. Having done so, the paper concludes by outlining a framework to
analyse one particular expression of epistemic activity; namely the deployment of e-
resources in SMEs designed to foster learning and knowledge creation.

The third paper, by Bernard Blandin addresses the topic ‘Localization of software and
learning material for SMEs: how is it possible?’ The paper revisits research undertaken
ten years ago through the COMETT programme (Auvinen, Blandin et al., 1994), which
aimed to identify cultural differences that have an impact on learning in order to facilitate
the localization and adaptation of learning material across boundaries. This approach
appears to be still valid, but, says Blandin, should be deepened, since further research
conducted in the meantime and sociological concepts such as the “social world”
(Blandin, 2002a) and “learning culture” (Blandin, 2003) are likely to help refine the
approach. It is not enough to adapt content: the “form” of the learning material has also to
be localized. Designing effective learning environments has to take into account social
and cultural factors, but also learning software and learning material usability has to be
considered as “situated” (Blandin, 2003). Occasions and conditions for the use of
learning material arise directly out of the context of learning activities which are
implemented.
Blandin asserts that the localization process as well as the design process for software
and/or learning material cannot be isolated from the design or the adaptation of learning
situations in which the software or the learning material is to be used. The paper presents
background research on cultural aspects of learning and their particular relevance to
SMEs, then goes on to examine how the “situated usability” of software and learning
material impact on these results. Finally, it proposes a two-tier development process for
the successful design and/or localization of software and learning materials.
The paper looks at learning styles, identifying an ‘Entrepreneurial City’ learning culture
and learning style for SME managers and employees. Blandin concludes that it is of the
utmost importance that e-learning materials are inserted in the right context of use, which
has to take into account the learning culture and the learning style of the ‘Entrepreneurial
City’.

The importance of local context is taken up in the fourth paper, ‘e-learning Content and
Software Localization’, by George Bekiaridis. The paper says that besides technology
itself, the major factors that impact on the successful implementation of vocational e-
learning programmes are content and software. The majority of e-learning content
produced globally is in English. However, especially for technical subjects there is
demand for content in native languages. It is obvious that content localization is
becoming ever more necessary in Europe.
The paper explores a number of the issues that influence the localization of vocational e-
learning content and proposes methods for the effective localization of e-learning.

11
Bekiaridis says localization is the process of adapting a product or service to a particular
language, culture, and desired local ‘look-and-feel’. Ideally, a product or service is
developed so that localization is relatively easy to achieve - for example, by creating
technical illustrations for manuals in which the text can easily be changed to another
language and allowing some expansion room for this purpose. This enabling process is
termed internationalisation. An internationalised product or service is therefore easier to
localize. The process of first enabling a product or service to be localized and then
localizing it for different national audiences is sometimes known as globalisation. The
paper looks at the different stages in localisation of software including technical
translation and cultural adaptation.

The fifth paper by Graham Attwell and Mike Malloch considers ‘Prospects for the
development of software that truly supports collaboration and learning in SMEs’. Most
attempts to develop ICT-based learning have failed to develop appropriate new models of
pedagogy and curriculum development. Indeed, it might be argued that most ICT-based
learning has been regressive in promoting more rigid, course-driven learning. Why
should this be so? The authors suggest that the major problem lies in a failure to
understand the relationship between the media being employed - predominantly learning
platforms and groupware technologies - and the issues of pedagogy and curriculum. All
software platforms have an inherent pedagogy built into their fundamental design
constraints and decisions. Different platforms support (or fail to support) different ways
of learning. In many cases ICT is used most effectively simply for information exchange,
and when learning does take place it does so outside the special ICT based e-learning
technologies provided.
There is a need to look more closely at how different software platforms and applications
support learning, and to develop a better understanding of the pedagogic processes which
take place when learners engage with these technologies. In the paper Attwell and
Malloch examine the use of collaborative environments for knowledge sharing, and ask
how existing software environments (especially software used for communication within
enterprises) support collaboration. They go on to discuss what kind of systems might be
developed to strengthen knowledge sharing and learning within workplace environments.
They say they are not suggesting that the development and implementation of new
software can by itself transform the workplace into a rich learning environment. But they
are suggesting that there is a need for software developers to work together with SMEs
and SME associations to iteratively develop and test new software tools and systems that
can facilitate collaborative knowledge sharing and development in real-world work
contexts. They contend that the focus on e-learning software and platforms (and the hype
around e-learning) has diverted attention away from the learning potential of email and
groupware products that are being used to a greater or lesser extent in larger corporations
and in some SMEs. They secondly argue that because of the ‘distance’ between
developers and users there has been little attention paid to the evaluation of software - not
in a technical sense, but in terms of its potential for enabling knowledge development and
learning.

The sixth paper, ‘A Framework for the Evaluation of E-learning’, by Jenny Hughes and
Graham Attwell looks at different theories, strategies and practice in the evaluation of e-
learning. The authors point out that e-learning has been an area that has attracted
considerable research and development funding. If this investment is to be maximised, it
is imperative that robust models are generated and used for the systematic evaluation of
e-learning. Evaluation tools are required, which are flexible in use but consistent in

12
results. Hughes and Attwell quote the American Society for Training and Development
(2001) in saying:

“Although recent attention has increased e­learning evaluation, the current research base 
for evaluating e­learning is inadequate…Due to the initial cost of implementing e­
learning programs, it is important to conduct evaluation studies.”

The paper is written from the perspective of professional evaluators and evaluation
researchers, rather than VET researchers or even e-learning practitioners. It is
underpinned by a set of assumptions about the evaluation process and also based on
personal research evidence that suggests that standards are more likely to be improved by
diversity, flexibility and experimentation than through standardisation. The first section
outlines the principles and assumptions on which their evaluation work is based. The
second section is a review of over 200 evaluation reports on e-learning with the focus on
identifying the type of report rather than its content. This is followed by a proposed
framework for reporting on and classifying e-learning evaluation together with some
lessons they have learned from their own evaluations. The overarching conclusion
reached by Hughes and Attwell is that the evaluation of e-learning is fundamentally the
same as the evaluation of any other learning, but with particular groups of variables
playing a more prominent role and the impact of others differing significantly from their
impact in traditional learning. Specifically, they suggest that political factors are crucial
because the nature of e-learning challenges conventional theory and established
benchmarks.

The seventh paper, ‘Practical evaluation interventions in understanding informal

learning within SMEs’, by David Slater, continues the evaluation theme, focussing on
approaches and methodologies for evaluating informal learning in SMES. Slater starts by
quoting Billett (2001) who suggests that informal learning is the primary form of
vocational learning – both initial and ongoing – for most people in the workforce today.
This is presumed to be especially true within smaller organisations. The paper looks at
the application of the work of Argylis (1965) when evaluating company culture and
internal interactions. It goes on to examine the work of Kleiner and Roth (1997) in
mapping organisational, as opposed to individual, ‘Learning Histories.’ Slater examines
the practicality and relevance of these theories and approaches to evaluating informal
learning in SMEs. He proposes there are three main stages that can be explored in
attempts to increase the role of informal learning in SMEs:

1. Identify those barriers currently in place. It is suggested that this be done

through exploring the ‘theories-in-use’ and ‘espoused theories’ currently
governing behaviour. However, other methods may be more appropriate
depending on circumstances.
2. Explore interventions that will serve to overcome those barriers. These
should be reviewed on an ongoing basis and moderated as appropriate.
3. Put in place mechanisms for the ongoing review of informal learning within
the SME.

It is recognised that an SME cannot normally be expected to spend considerable time or

resources on these activities, save as a response to recognised and acute difficulties.
However, where relatively simple mechanisms can be put in place, requiring the

13
minimum of external support, SMEs can considerably boost the potential for informal
learning to take place among their staff.

The eighth paper, by Eduardo Figueira, is on ‘Evaluating the effectiveness of e-learning

strategies for SMEs”. Figueira says that the development and use of any e-learning
programme or strategy represents a considerable individual, organisational and social
investment. For this reason, the effectiveness of any e-learning provision should be
evaluated. Without knowing the efficacy of e-learning strategies, one cannot know if it is
worth using them or not. So, measuring effectiveness constitutes a very useful tool for
basing decisions on the adoption of any e-learning strategy. Built-in programme
evaluation allows trainers and others responsible to monitor e-learning programmes and
provision and make changes for improvement.
Globally, programme effectiveness can be evaluated using five types of approach:
1. Based on the programme goals,
2. Based on the decision-making process,
3. Goal-free,
4. Based on an expert’s knowledge, and
5. Naturalistic approach.
Figueira proposes a global framework for evaluation comprising a conceptual matrix to
guide development of criteria and indicators for the evaluation of the effectiveness of e-
learning. The framework, intended to be utilized for measuring the effectiveness of e-
learning strategies, is based on a revised version of Bennett’s hierarchy and measures
three main levels: participation, reactions and results.
In the final part of the paper Figueira proposes criteria to evaluate effectiveness of e-
learning. He says it appears that measuring the effectiveness of e-learning should take
into consideration the nature, characteristics and objectives of the e-learning strategies.
For this reason, approaches to the evaluation of the effectiveness of e-learning should
take into account the following e-learning dimensions:
1. Organisational & Management dimensions,
2. Pedagogical dimension,
3. Technological dimension,
4. Ethical considerations,
5. Learning assessment and certification, and
6. Evaluation Strategy.

Paper nine by Klaus Reich and Friedrich Scheuermann presents a case-study of ‘E-
learning challenges in Austrian SMEs’. They say e-learning is changing the way
enterprises gain competitive advantage through improved human performance. Small and
medium-sized enterprises have the problem that e-learning technologies, methods and
strategies have mostly been developed to meet the needs of large enterprises and cannot
be exactly transferred to their needs. Based on a survey by the authors of e-learning in
Austrian SME’s (Scheuermann & Reich, 2002), the current state of ICT based learning in

14
Austria is analyzed and suggestions for better implementation and use of e-learning are
made, with a special focus on cooperative e-learning approaches.
If better e-learning solutions are to be found, Reich and Scheuermann suggest that groups
of SMEs have to be identified. Cooperation and partnership are essential because it is too
expensive to develop solutions for individual SMEs. SMEs have to develop new
organisational structures for the implementation of cooperative and collaborative forms
of learning. At the moment most SMEs implement isolated learning solutions that fail to
meet the needs of cooperative and collaborative forms of working and learning. New
approaches are needed in terms of understanding the broad context of learning.
Pedagogical approaches in Austrian SMEs are often inadequate and do not meet the
needs of e-learning. It is also suggested that there is a lack of flexible learning solutions
available to SMEs. There is still much to be discovered about how people learn using
different technologies, particularly in relation to interactivity, and how materials can be
developed and structured to enable all learners to make effective use of them.
Learning at the workplace should partly replace ‘old’ teaching in classrooms and face-to-
face courses away from the enterprise, but this is difficult to implement. Integration is
important: learning should not take place besides work, but with and through work.
Learning at the workplace is different from learning at school and in a classroom. A
problem-based approach is required: rather than using a pedagogic models based on
linear progression through subject knowledge it is more relevant to base learning in the
solution of problems arising from practice. Flexible learning media are needed.

Paper ten, by John Munro and Philip Crompton, is entitled ‘Assessing the Application of
Online Learning in a Work-based Setting’. Munro and Crompton say globalisation has
increased the pressures on companies and national economies towards greater
competitiveness. Much faith has been put by governments and educational policy makers
in the development of knowledge economies where information becomes the key
component for creating competitiveness. This has led to increasing interest in the
development of work-based learning, lifelong learning and the application of Web-based
learning, or e-learning as it has come to be known, as mechanisms for bringing learning
into the workplace, the home and the community.
Munro and Crompton provide a case study of the e-learning experience of a medium
sized enterprise in Central Scotland. The purpose of the investigation was to assess the
impact of online learning in a work-based setting and to determine if the course content
was appropriate to the learning needs of the employees. The company piloted a course for
51 employees based on the Finance Module of the Certificate in SME Management
(CSMEM). The module was provided in online format by the University of Stirling. The
case study identifies several pertinent issues relating to the role of e-learning in the
workplace, including the practicalities of learning by ICT, time and workload pressures,
company culture, and the learning preferences of the employees.

The final paper, ‘A holistic vision of the future of e-learning’, by Kees Schuur, provides
a wide-ranging discussion of the future of e-learning. In his introduction, Schuur says e-
learning environments have become readily available during the last 5 years. In addition
there is a great deal of ‘hype’: if we believe the claimed advantages of e-learning, for
example cost savings, Just-In-Time learning, at the learners’ own pace, at any place, any
time, flexible learning styles, etc, the future looks bright. Despite this, only a small
percentage of learners, especially in industry, are currently using e-learning
environments.

15
Schuur explores several factors that influence the use of e-learning environments from
the point of view of developments in different disciplines and in society, and provides a
vision of the future for ICT based learning. It outlines for development of e-learning
environments in the future. It is not intended to be an exhaustive list, but rather to show
the direction of development and to be a starting point for further discussion. Examples
of different systems and platforms are included to illustrate the discourse.
What kind of impact will new technologies have on society? Where are we heading?
How can we shape the development of technology? And what will the impact be on our
learning and the instruments we use for learning, such as e-learning?
Attempts to predict the future are always perilous. Schuur points out that there are at least
three ways of forecasting:
(1) One way to create a picture of the future is to identify present technologies and based
on this to describe possibilities for the near future.
(2) A second way is to look back at historical developments and to extrapolate from these
into the future.
(3) A third way is to combine elements from different disciplines and view different
combinations as pictures of the future.

Common Themes:
There are a number of common themes that emerge from the papers and the discussions
that took place in the Stirling and Brussels seminars:

• The importance of informal learning and the need to take this into
account in the design and implementation of e-learning.

• The implications of constructivist theory with its emphasis on

collaborative learning.

• The need to integrate e-learning with other social and cultural

processes, implying local adaptation.

• The requirement to evaluate the effectiveness of e-learning before

embarking on its whole-scale implementation.

• The nature of the knowledge required in SMEs and the significance

of a move to knowledge management.

• Disappointment with what appears to be the relatively slow rate of

uptake of e-learning (but we tend to forget how recent a phenomenon the
Web is).

• The need to follow the logic of partnership in the design and

implementation of e-learning strategies. Individual SMEs are unlikely
to be able to afford the investment required to develop effective e-
learning strategies by themselves: what is needed is a community of
learning practitioners.

16
The papers published here are intended to be the first step in the development of such a
community.

References
American Society for Training and Development (2001)
Argyris, C. (1965) Organisation and Innovation, Homewood, III: R.D. Irwin
Attwell, G. (forthcoming) The challenge of eLearning in small enterprises: issues for
policy and practice in Europe
Auvinen, A.M. Blandin, B. Chapman, P. Dondi, C. Evans, R. Goldstone, L. Sharratt, R.
edited by Johnstone, A. (1994). Cultural impact on learning : A practical guide to
managing the effective adaptation of learning materials across international boundaries.
Billett, S. (2001) “Participation and continuity at work : A critique of current workplace
learning discourses. Context, Power and perspective : Confronting the Challenges to
Improving Attainment in Learning at Work”. Joint Network / SKOPE / TRLP
International workshop 8-10th November 2001, Sunley Management Centre, University
College of Northampton. Available at the informal education archives:
http://www.infed.org/archives/e-texts/billett_workplace_learning
Blandin, B. (2003) Usability Evaluation of Online Learning Programmes: a Sociological
Standpoint”, in Ghaoui, C. (ed). Usability Evaluation of Online Learning. Idea Group
Inc. (PA), pp.313-330.
EU (2000), The Lisbon strategy for economic, social and environmental renewal,
http://europa.eu.int/comm/lisbon_strategy/index_en.html
Guile, D. (2001) “From ‘credentialism’ to the ‘practice of learning’: rethinking learning
for the knowledge society”, Policy Futures in Education, 1, 1.
Kleiner, A. and Roth G. (1997) Learning Histories. A New Tool for Turning
Organisational Experience into Action. http://ccs.mit.edu/LH/21wcp002.html
Knorr Cetina, K. (1999) Epistemic Communities, Boston: Mass:Harvard Education Press
Lundvall, B.-Å. “The Social Dimension of The Learning Economy,” DRUID Working
Papers 96-1, DRUID, Copenhagen Business School, Department of Industrial Economics
and Strategy/Aalborg University, Department of Business Studies .
http://ideas.repec.org/p/aal/abbswp/96-1.html
Scheuermann, F. and Reich, K.(2002 in press): ELearning in Austrian SMEs. Innsbruck,
2002.

17
18
Developing new pedagogies and learning in SMEs

Graham Attwell, KnowNet (UK)

European seminars
In mid 2002 KnowNet, a small research company based in North Wales, were awarded a
grant by the European Commission DG Education and Culture to host two seminars
around the theme of ICT and learning In Small and Medium enterprises. The seminars
were intended to build on the experience and outcomes of existing European projects and
research in the field in order to focus on the issues for developing the use of ICT for
learning in SMEs and produce policy recommendations for the European Commission
and Member States.
The first seminar was help in Stirling, Scotland in November 2002 and the second in
Brussels in February 2003
Whilst the themes for the seminars were broad they the seminars focused in particular at
the issue of pedagogies for learning in SMEs and at the evaluation of learning using ICT
in SMEs. This included the following issues:
• How can we measure the effectiveness of e-learning in SMEs?
• How can we measure the effectiveness of different models for e-learning in SMEs
• What constitutes a rich learning environment for e-learning?
• How do different social factors impact on learning in SMEs?
• What pedagogic models are being used for e-learning in SMEs?
• What pedagogies can support the use of ICT for e-learning in SMEs
• What is the relationship between different pedagogies and different technological
platforms?
• What is the role of the teacher or trainer in supporting learning using ICTs in
SMEs?
• What is the relationship between changing business practice and e-learning in
SMEs?
In total twenty-six researchers attended the two seminars, drawn from nine different
European countries. They were selected for their previous work in this area. The seminars
were planned with a dual agenda. Although the programme provided for the formal
presentation of research papers, this was accompanied by exploratory workshop activities
designed to encourage reflection on the issues and problems and to develop policy
recommendations.
Two parallel workshop strands were developed – one around pedagogies and e-learning
in SMEs and the second around evaluation and e-learning in SMEs. This paper provides
an account of the workshop activities on pedagogies and SMEs. It is not intended as an
academic or research paper as such, but rather as providing access to the broad ranging
discussions that took place.

Definitions of e-Learning and SMEs

The issue of what comprises an SME is not just a semantic issue. The major issue raised
in discussion was that the needs of SMEs depend on a number of different factors of
which one of the most important is size. The capacity for building the infrastructure and

19
the capability to develop an appropriate learning cultures to support e-learning will be
very different in micro enterprises (<50 employees) than in larger SME. Similarly, there
will be differences between enterprises of around 50 – 100 employees and enterprises
between 100 employees and the official upper level of 350 employees. The feeling of the
group was that the greatest need for research and development was in the micro SME and
in those with between 50 and 100 employees as these are the groups who have the most
limited resources and thus experience the greatest problems in providing training for
employees.
The definition of e-learning is very important in this context. The development of e-
learning has been driven predominantly through the application of the virtual classroom
paradigm for academic learning in the higher education sector. Thus, it has taken on
many of the connotations of traditional learning in this environment. e-learning has
tended to be seen as the use of digital media (including CD-ROMs, the World Wide Web
and intranets), for the delivery of structured programmes which are materials based. This
definition is problematic.
Many SMEs make extensive use of digital media for accessing technical manuals and for
web searches and increasing numbers are involved in business to business (B2B) e-
commerce applications. This is often taken as evidence that they are engaging in e-
learning. However, we would challenge the idea that the use of ICT for information
storage and retrieval necessarily constitutes learning. Rather, we would contend that for
real learning to take place, this new information has to be applied in a way which then
develops new mental models and schema, be they tacit or explicit. For collective
knowledge development to take place, these new mental models have to be made explicit
and shared. There is no doubt that CD-ROMs and the use of the web are useful and
convenient ways of storing and retrieving information but unless that information is
transformed through use it comprises neither learning nor knowledge development.
Nevertheless, e-learning in SMEs may be most powerful when it is integrated in company
business processes through networks and systems for commerce and business
development. The tools or software systems used in learning will often not be dedicated
e-learning platforms but everyday business systems and software. In fact, email may be
the most common learning tool for sharing information and new practices within the
workplace, leading to knowledge creation. Critically, the learning materials may well not
be bought in or prepared by e-learning specialists, software houses or multi media
publishers but the products of employees’ documented and shared enquiry into their own
practice.
This requires a far wider definition of e-learning than is used currently. It means taking
into account all the electronic and digital resources – both formal and informal – used by
the enterprise as potential learning and knowledge development tools. It also means
looking at the totality of business processes in designing the e-learning environment. In
this respect, e-commerce may be a considerable driver for change. For example, even the
smaller SMEs are increasingly using computers and software systems as part of their
business organisation for a range of different tasks and processes, including the
procurement of materials, logistic organisation, sales and marketing and process control.
It is possible that e-learning may be most effective when it is integrated with these e-
commerce practices and with the use of digital media, computers and networks within
work organisations and structures as it then becomes part of the culture of the enterprise.
Such a perspective breaks down the divide between what has tended to be seen as e-
learning and what has been categorised as `knowledge management’ or management

20
development. Pedagogically, it integrates the acquisition and practice of skills and
knowledge with work processes, developing what some researchers have referred to as
work process knowledge (Rauner and Bremer, 2001)
Barry Nyhan (2003) talks of “building organisations in which people have what can be
termed ‘developmental work tasks’.” These are challenging tasks that ‘compel’ people to
stretch their potential and muster up new resources to manage demanding situations. In
carrying out ‘developmental work tasks’ people are ‘developing themselves’ and are thus
engaged in what can be termed ‘developmental learning'.” The challenge for developing
e-learning in enterprises is the integration of ICT in such a way that it supports
developmental work tasks, rather than merely electronically cataloguing routine roles and
tasks.
Participants in the Stirling seminar said that e-learning is “Always combined with other
learning”. This is important. E-learning, whether in SMEs, large corporations or in
schools cannot be viewed as an entity in itself. This is a critical idea when it comes to the
design of learning environments for SMEs.

Managers attitudes
The analysis above is taken primarily from a research perspective. The scene changes
when we move to the perspective of the SMEs themselves - or more accurately the SME
manager or owner. Participants in the Stirling seminar spent some time discussing exactly
what SMEs wanted from e-learning and concluded that because there is very little
empirical research, much of this discussion was anecdotal.
Firstly, from our own personal experiences, SMEs want to break away from institutions
and institutional based training. Despite the much-flaunted move towards flexibility in
vocational education and training, VET institutions in general are still unable to meet the
needs of SMEs. Managers are often looking for not `just-in-time’ but `absolutely-last-
minute’ training. In order to provide this they need flexible learning materials and
flexible methods of delivery. Current technologies and processes for developing and
delivering materials are problematic in this respect. Arguably, this sort of training for
SMEs is not a core function for VET institutions and we discussed the need for processes
and systems, which allowed SMEs to develop and share their own materials (see below).
Thirdly, SMEs want to save money. They perceive e-learning as a potentially cheap and
effective form of last-minute training. Given the present economic recession in many
countries in Europe and increased competition, many SMES say they can no longer
afford face-to-face training. The perception that e-learning is cheap is not without
problems (see section on `costs’ below).
The difference between `needs’ and `wants’ is particularly important when it comes to
SMEs. Few SMEs carry out systematic training needs analyses; neither do they have
training officers. Few have full time qualified trainers. The CEDEFOP study (Attwell,
forthcoming) on e-learning and SMEs concluded that the attitude of individual managers
is the single most decisive factor in influencing the development of ICT for learning in
SMEs. Managers tended to be pragmatic in their wishes with regard to training rather
than following any scientific method of assessing needs.
The CEDEFOP study found little support by managers for introducing e-learning. More
surprisingly there was little support for SMEs or SME managers in developing e-learning
in their enterprise. A number of the companies in the, admittedly small, sample were
using e-Learning because of their role in supply chains. In Italy, regional organisations

21
were trying to stimulate the use of e-learning. In Birmingham, in the UK, the local
government was promoting e-learning through an EU financed ADAPT project, In the
Austrian Tyrol, the Chamber of Trade has some involvement. However, in each case the
support available was both quite specific and quite limited. There was no natural point to
which managers could turn for help, nor did the use of e-Learning appear to be a major
issue for the different support agencies and networks in which the SMEs were involved.

Cost benefits
The question of who pays is perhaps the most pressing, unresolved and complex issue in
the development of e-learning for SMEs. The issue is significant at a number of different
levels. E-learning has been widely promoted as a cheap or cost effective answer to the
policy issue of engendering and promoting Lifelong learning within SMEs. Yet, most
researchers, and many consultants, are adamant that e-learning cannot be seen as a cheap
solution. There may be economies of scale to be reaped in the future but even this is open
to doubt. Despite the many ROIs – or Return on Investment Studies – it is unclear
whether e-learning is at present competitive when compared with traditional learning
methods. E-learning must, therefore, be justified in its own terms – in opening up access
to Lifelong Learning or in providing richer learning environments for learning.
It was generally agreed that the prime task was to develop the efficiency and
effectiveness of e-learning and promote the benefits to SMEs. However, there were two
counter posed views, expressed within the Stirling seminar, on how this issue should be
addressed in the future. One was that as e-learning is too expensive for SMEs and is
likely to remain so for the foreseeable future, the state should subsidise e-learning by
fostering new ideas and supporting the implementation of new technologies in order to
provide access to new learning infrastructures and cultures. The second was that, despite
present low rates of take up, e-learning should be developed through the market.

Standards and technology and content

Access to learning materials is a major issue in the development and implementation of e-
learning in SMEs. As the European Commission notes in a recent document
“the development of the `digital economy’ and the wider use of the Internet and
of computer and networking equipment has raised accessibility to multimedia to
unprecedented levels, thus enhancing opportunities for producers and
consumers…However, this rapid evolution does not seem to have been matched
in the new educational content sector.”
The consensus was that there is a lack of European educational multimedia content
coming from institutional, professional and industrial sources in the education, publishing
and educational software sectors. After an initial phase of enthusiasm, often described as
“hype”, there are growing doubts about the real demand for educational e-content, and
about its relevance for improving learning”1. Our studies would, in general, bear out this
assessment.
An examination of an extensive catalogue of e-learning materials available in Italy has
revealed an extremely limited range of subject area provision. Most materials are for
technologies, mainly the use of standard software packages, and for networking. The next
1European Commission Directorate-General for Education and Culture, Open invitation to tender No DG
EAC 21/02 for the provision of services concerning the carrying out of studies in the context of the
eLearning initiative.

22
largest category was learning materials for managers and for management activities such
as marketing and thirdly, materials for language learning. Beyond this the provision is
very limited. Obviously these materials are largely targeted at technical, professional and
management employees (or white collar workers).
The situation is further exacerbated in Europe because of the diversity of languages. E-
learning was pioneered in Europe in the university sector and most learning materials
were provided in English. Whilst this may be acceptable in a higher education
environment, most vocational learners require learning materials in their own language.
Dondi and Zucchini (1998) have drawn attention to the problems of market-led software
and materials being provided in different language versions, especially those languages
serving smaller population groups. Furthermore, whilst universities could draw on very
broad discipline areas and on common discipline structures between countries, the
learning needs of employees in SMEs are often quite narrow and very specific, with
limited international (or even national) markets. There is a growing debate on the issue of
globalisation and localisation in software and learning materials and the group agreed that
the task of translating materials to different languages is not a straightforward technical
issue but involves significant cultural transformations.
There have been many policy initiatives and measures to stimulate market provision of e-
Learning materials. A considerable number of projects have encouraged universities to
provide learning programmes and materials for SMEs. However, it is open to question as
to how effective these initiatives have been – and to the sustainability of project led
developments.
One of the key debates revolves around standards, both for learning materials and
learning platforms. Standards mean that learning materials and systems would be
interoperable – and therefore more sustainable. Standards would enable and facilitate the
development of learning materials within an individual enterprise or cluster of
enterprises. Within a wider vision of e-Learning, that is, e-learning as a contribution to
knowledge management or collective knowledge development, a learning ‘object’ or
discrete piece of learning material is seen as the fusion of the outcomes of learning
together with the knowledge created, at all its different stages and in all its different
forms. Learning objects or materials could be created by the learner themselves as a
result of their engagement in business activities. The primary role of the computer based
learning platform would not be in the delivery of the materials but in facilitating the
transformation and communication of ideas as knowledge. The computer or ICT based
learning environment would be primarily a process tool to support the creation and
transformation of knowledge. The overwhelming advantage of this approach is it allows
learning materials to reflect and support the different contexts in which learning takes
place in SMEs.
The development and implementation of learning platforms is yet another difficult issue.
In universities early development tended to be through the use of platforms which were
locally produced and maintained. More recently, the trend has been towards adoption of
(increasingly technically sophisticated) platforms developed by private (mainly
American) e-Learning technology specialists. Over the last year, as the market situation
has become tighter, there has been a spate of takeovers and mergers, accompanied by
‘downsizing’ in those remaining companies.
Participants in the Stirling seminar were sceptical that technological advance has been
accompanied by improvements in the pedagogies that these platforms facilitate. The cost
of server software applications, let alone the difficulties in installing and maintaining
server-based systems, is beyond the reach of most SMEs. This leaves them a limited
range of options. CD-ROMS, although useful in some contexts, do not allow

23
communication between learners. Buying off-the-shelf courses from providers operating
their own platforms and servers restricts choice and there is little or no opportunity for
SMEs to develop their own learning materials or to manage and develop their own
knowledge systems. Once again, the obvious solution is for clusters or groups of SMEs to
collaborate on a regional basis, with umbrella organisations or networks offering software
platforms as well as learning materials. We have nothing against standardised
commercial learning platforms. Our major point is that the learning technologies and
systems should form part of wider knowledge sharing and development networks. It is
also very important that they support the emergent standards for e-learning materials in
order to allow interoperability and sharing of development efforts, and to facilitate
localisation.
New paradigms
Towards the end of the Stirling group discussions, participants started looking at the
problem of forecasting future developments and what the implications might be for
SMEs. We agreed that the real challenges were to stop thinking within existing
paradigms of learning and e-learning and to develop new ones. Whilst this discussion was
inconclusive and a little fragmented, it raised a number of interesting issues.
One was the nature of informal (or tacit) knowledge as opposed to formal knowledge. E-
learning until now has concentrated on formal learning and the reproduction of traditional
courses through digital media. Yet most studies suggest that, at least for skilled work, it is
the tacit knowledge that is most powerful within SMEs. The question is how e-learning
systems and architectures can be used to support the development and sharing of tacit or
informal knowledge.
The second issue was the tension between individual learning, which e-learning tends to
be, and the social nature of more traditional learning. The social dimension is recognised
as an important motivator and driver for many learners and a positive social environment
can improve the effectiveness of learning as well as allowing a wider range of
methodologies.
Thirdly, if the sharing of knowledge and the development of new knowledge are
important future developments for SMES and if e-learning or e-resources are to be a
medium for doing this, we need to look at the nature of the interactions in e-learning and
to develop new processes and solutions. We also agreed on the need to move towards a
new paradigm of continuous learning based on the application of new working principles.
Fourthly, but arguably more fundamental, was the question of what problem is e-learning
trying to solve. Is it of benefit for SMEs at the present stage of technical and pedagogic
development? Clearly SMEs are using ICT technologies for access to information but
they see that as very different to signing up for e-learning ‘courses’. It was said that some
SMEs think they need e-learning but they do not necessarily know why! In many cases it
is being driven by the need to employ new information technologies but the lack of any
integration with e--earning is holding back progress.
Fifthly, there is a separate issue around how to persuade employees themselves, rather
than their bosses, to use new technologies for learning and how are individual learning
needs reconciled with collective ones. That is, how can we resolve the PSP paradox -
providing pleasure, sustainability and profit all at the same time! We were unconvinced
that we knew the answer and doubted there was one.
Brussels seminar

24
The second EC sponsored seminar was held in Brussels in February 2003. The seminar
was designed to build on the work of the first meeting in Stirling. Following a revue of
the outcomes of the first seminar, participants were asked to develop a series of policy
recommendations, if possible directed at local, national and European level. The aim was
to transcend the gap between research, policy and practice.
Learning cultures
The first session reviewed the Stirling outcomes. It was felt that not enough attention had
been paid to the learning culture in SMEs. The learning culture in SMEs differed
significantly from that in larger organisations. SMEs had a different value system. SME
managers, and employees in SMEs had a stronger identification with the enterprise and
with the role the enterprise plays in local and regional economies. Whilst larger
companies were driven by maximisation of profit, other goals, such as developing local
employment, were important to SMEs. Knowledge is developed and shared in different
ways in SMEs. Whilst in larger companies there are formal procedures and processes for
the sharing of information and knowledge within the organisation, informal processes and
peer group networks and local communities were often more significant in SMEs.
The culture of SMEs is characterised by strong interpersonal networks, both within the
enterprise and between different enterprises. Clubs and associations play a key role in the
culture of local SMEs and are frequently a central source of ideas and new knowledge.
At the same time it was recognised that there are important barriers to the sharing of
knowledge between SMEs. Knowledge may be seen as providing competitive advantage
in local economies, particularly between companies in the same sector. This was seen as
a complex issue. Undoubtedly there are some issues and challenges, which are shared
between different SMEs and enterprises, are happy to work together to respond to these
issues. However, in other cases sharing knowledge would be seen as helping competitors.
In this scenario informal learning plays a far greater role than in the more formally
structured larger organisations. Learning is more likely to take place as a result of
challenges to practice and equally more likely to take place within peer group interactions
or through communities of practice. This has significant implications for the design of e-
learning systems for SMEs.
Regulation and certification
The issue of regulation and certification caused considerable discussion. Certification of
learning was seen as a cultural issue. In some sectors in some occupations in some
countries certification of learning was a vital issue. But in other sector and countries
certification of learning was not seen as a pressing issue, more important was the learning
and experience gained. The regulation of learning and training could lead to a formal ‘e-
market’ for learning – but at the same time could prescribe the form and nature of
learning taking place and fail to recognise and support the importance of informal
learning.
Possible solutions
The second session in the Brussels seminar began to consider possible solutions to the
issues that had been identified. It was seen as important to create “Cultures of Learning”
within and between SMEs. In this regard it is necessary to identify and involve the
different stakeholders who may come from quite diverse contexts and backgrounds. A
major question was as to whether it is possible to introduce e-learning from outside the
culture of the local and regional SMEs. Whilst many universities and training
organisations in Europe had tried to promote the virtues of e-learning, they were coming
in from the outside and often did not relate to the culture of the target groups. It was often

25
difficult for those outside the SME culture to understand the needs and concerns of
SMEs. Even language could be a problem with SME managers unable to relate to the
language and use of language by researchers and ICT based learning developers.
In looking at potential solutions it was important that e-learning was not approached as an
issue or solution in itself. The discussion over e-learning for SMEs was only an
‘amplification’ of the overall question of the provision of learning and training in SMEs.
E-learning would not always be the most appropriate form of learning and would nearly
always need to be complemented by other forms of learning using more traditional
technologies.
It was not enough just to adapt learning materials for use in SMEs. We need to adapt the
entire learning environment and the learning models we deploy – what one participant
called “engineering of the form”.

Actions for pedagogy

The final session of the workshop programme focused on actions and recommendations
for actions to develop new pedagogies and promote the use of e-learning in SMEs. Once
more, the discussion was wide ranging.
Promote Cultures of Learning
The most critical issue was to promote a culture of learning in SMEs. This entailed taking
actions to involve a wide range of stakeholders including training organisations and
providers, local and regional government organisations, SME associations, regional
economic development organisations, sector organisations, employers organisations,
trade unions and professional bodies, supply chains and community bodies – including
vitally those clubs and societies to which SME managers and employees belonged – as
well as SMEs themselves. This was seen as a long-term strategy but one without which
no amount of e-learning directed initiatives would succeed.
Mobilize SME managers
As part if such an approach it was particularly important to mobilize SME managers
through the development and support of sector networks, regional networks and regional
learning networks.
Focus on organisational development
E-learning could not be viewed as an issue or aim in itself. In fact the present focus on
learning and training was even seen as unhelpful, particularly given the vague and
unfocused nature of discourses on lifelong learning. Instead e-learning had to be
addressed within the context of organisational development, including more support for
SMEs in introducing new forms of work organisation and new technologies within the
workplace.
Support networks as a new developmental paradigm
Networks should be seen as a new developmental paradigm. Present approaches to
learning from policy makers and planners still reflected an ”industrial age model” with
lifelong learning merely seen as “more of the same for more people”.
Develop a differentiated approach
Such a new approach needed to be more culturally differentiated – “one size does not fit
all”. Due recognition of the different needs of different sectors and regions is necessary –
rather than a blanket approach to introducing e-learning in SMEs.

26
Recognise triggers for change
It was also important to identify the different triggers that acted as a catalyst for change
and to provoke the promotion of learning and a learning culture. These would vary again
between sectors and regions and over time. When these trigger points were recognised
structures were need to take immediate short-term actions to capitalise on opportunities.
Policy support
There were many actions that could be taken by local, national and European
policymakers to support the use of ICT for learning.
Participants proposed a change in present funding policies to promote and support larger
numbers of small projects based on communities of practice and running more
intensively over shorter time periods. These projects should be focused on work based
learning rather than of the provisions of distance learning courses or on virtual
classrooms and should seek to develop and capitalise on informal processes of learning.
Rather than look to external learning materials developers or training bodies to supply
learning programmes they should focus on SME employees as providers (or senders) of
knowledge. The development of competence in SMEs was seen as predicated on the
knowledge, skills and attitudes of the SME employees themselves.
Present programme and project based evaluation was too slow and focused in the wrong
areas. Instead policy makers should be developing the skills and culture of on-going,
continuous self-evaluation based on communities of practice.
At a European level inter-regional networks and projects were seen as particularly
valuable. Similarly, projects support for clusters of SMEs offers great potential.
For a number of years researchers and developers have been drawing attention to the
issue of standards and interoperability. The European Commission should urgently
address this issue. One possible policy would be to insist that all publicly funded projects
must be based on Open Standards software and outcomes must be available through the
General Public Licence.
A further change in policy and funding could be to provide more support for the different
industrial associations – chambers of commerce and sector organisations that represent
SMEs – although there was some concern that they might prove too bureaucratic to lead
the changes we sought to promote.
The final discussion was around the future priorities for research in this area. Three main
themes were identified.
The first was to research how SME managers and SME employees are using ICT within
their everyday work and to use this as a basis for developing environments and
opportunities for e-learning.
The second was to recognise and ongoing paradigm shift from e-learning to knowledge
management and to initiate research on knowledge management including ‘How does it
happen?’ - ‘What technologies support it?’ - and ‘Chaos management, complexity
management, and the management of uncertainty”.
The final need was for more research into social networks and SMEs and their impact on
knowledge sharing, development and learning.
Participants recognised the need for new forms of research and in particular to promote
‘accompanying research’ – with research taking place alongside developmental activities.
This could help overcome the present divides between research and practice and
researchers and practitioners. It was suggested that if the European Commission was to
see fit to support future seminars of this type, it could prove fruitful to bring
representatives of industrial associations and SME managers to participate along with
researchers from the field.

27
References
Attwell G. (forthcoming), The challenge of e-learning in small enterprises: issues for
policy and practice in Europe. Luxembourg: Office for Official Publications of the
European Communities.
Dondi C. and Zucchini I., “Economic and Organisational Issues in the Trans-National
Development and Delivery of ODL and eLearning Courses”, in Research Perspectives
on Open Distance Learning, SCIENTER, Bologna, 1998
Nyhan, B.; Cressey, P.; Tomassini, M.; Kelleher, M.; Poell, R. (2003) Facing up to the
learning organisation challenge, Volume I – key issues from a European perspective.
Luxembourg: Office for Official Publications of the European Communities.
Rauner, F. & Bremer, R, (2001) Experten-Facharbeiter-Workshops als instrument der
brufswissenschaftlichen Qualifihationsforschung’ ‘Mensch-Maschine-Inetraktion’
Arbeiten und lernen in rechnergestuetzen Arbeitessystemen in Industrie, Handwerk und
Dienstleistung, Eicker, Friedhelm A; Patersen W. (eds), Baden-Baden: Nomos

28
29
Epistemic activity, organisations and learning:
towards a framework for analysing the use of e-
resources in small and medium enterprises

David Guile, University of London

The Learning economy

Discussions in EU policy about the ‘knowledge economy’ have tended to emphasise the
contribution the technological output of knowledge makes to economic development, and
the use of ‘e-learning’ to widen access to education and training so that more people
acquire the qualifications needed for ‘knowledge work’ (EU 1996). This focus is
certainly understandable since policymakers, along with educational researchers, have a
longstanding concern with the relationship between qualifications, employment and
economic success. However, by focusing on the increase in knowledge-based products
and services and on the demand for qualifications, policymakers and educational
researchers have tended to overlook certain issues as regards the concept of the
knowledge economy and its relationship to learning and the use of e-resources2.
The first issue is that the knowledge economy does not just hinge on the increased
presence of technological and informational products of knowledge and on the volume of
qualifications. It also means that ‘knowledge cultures’ (Knorr Cetina 1997) have spilled
and woven their tissue into the fabric of society and workplaces. A knowledge economy
is not just an economy of more experts, more technological gadgets, more specialist
interpretations any more than lifelong learning is primarily concerned with the
accumulation of more qualifications. On the one hand, knowledge economies presuppose
the presence of knowledge cultures, that is, the ‘whole set of structures and practices that
serve knowledge and unfold with its articulation’ (Knorr Cetina (1997:70). On the other
hand, they also presuppose constant renewal and transformation, in other words, they
presuppose the creative use of material artefacts (e.g. computers) and symbolic artefacts
(e.g. languages) to facilitate learning and knowledge creation.
The second issue is that although the level of educational qualifications workers hold is
an important issue, qualifications are not the determinants of ‘knowledge work’ (Guile
2001). Knowledge work presupposes the distinctive forms of social organisation and
social practices, which enable people to learn to produce, circulated and use knowledge
in workplace. They also presuppose constant renewal and transformation, that is, they
depend upon an individual and collective capacity for learning.
The aim of this paper is to provide a framework for analysing the symbiotic relationship
between knowledge cultures, learning and e-resources. In order to do so, the paper
introduces a new concept - ‘epistemic activity’. The concept of epistemic activity refers
to participation in social practices, including the use of their associated artefacts, which
facilitate the production of and application of new knowledge. The paper starts by
identifying the premises that inform the concept of epistemic activity; it then proceeds to
use the concept to analyse the symbiotic relationship between knowledge cultures and the
production of new knowledge. Having done so, the paper concludes by outlining a
framework to analyse one particular expression of epistemic activity; namely the
2 The paper uses the term e-resources instead of Information and Communication Technology to reflect the
idea that such technologies constitute, in theory, a single resource which can be deployed to enhance
working and learning.

30
deployment of e-resources in Small and Medium Size Enterprises to foster learning and
knowledge creation.

The concept of epistemic activity

Background to the concept.
The concept of ‘epistemic activity’ is based on recent work in two very different branches
of the social sciences. The first is work in the Sociology of Science/Social Theory, which
has turned the conventional argument about knowledge societies, namely that knowledge
is a component of economic and technological paradigms on its head. Knorr Cetina
(1999, forthcoming) has argued that the dehisence of knowledge has resulted in the
growth of ‘epistemic (i.e. knowledge) cultures’ in an increasing range of contexts
throughout advanced industrial societies: a development that implies that social,
economic and organisational life may become part and parcel of particular knowledge
cultures. Moreover, she claims that this development implies a more ‘inclusory’ approach
as regards access to knowledge in all types of organisations. The second is recent work in
Cultural Historical Activity Theory which has argued that, in order to understand the
practice of knowledge creation, it is important to understand how ‘culture weaves things
together’ (Cole 1996). The next section of the paper discusses the contribution of both
theoretical traditions to understanding the symbiotic relationship between knowledge
cultures and knowledge production. It concludes by arguing that their respective
assumptions constitute the theoretical basis for the concept of epistemic activity.

The idea of epistemic cultures. 
Historically, epistemic cultures have always been associated with science. One of their
defining features was the existence of an ‘experimental system’ that facilitated a process
of reasoning which result in the production of new knowledge (Rheinberger (1992).
Experimental systems, according to Rheinberger, were characterised by two distinct, but
related, activities: ‘question generating’ and ‘answering providing’ activities
(Rheinberger 1992). The former refers to the practice of identifying an object of inquiry,
in other words, the ‘epistemic’ thing’ to be investigated through the research. The latter
refers to the ‘technological object(s)’, that is, the stable moments of experimental
research that are achieved through the establishment of criteria to guide the investigation,
to structure the allocation and use of those artefacts involved in the investigation,
normally in laboratories, and to determine the pattern of working.
The distinction between the ‘question generating’ and ‘answering providing’ activities is
an analytical and not a material distinction. It does not define fixed parts of a system,
rather it reflects ‘places’ within it and how the practice of ‘question generating’ and
‘answering providing’ may change places with one another from time to time
(Rheinberger 1992).
According to Knorr Cetina, Rheinberger’s idea’s about epistemic things, which
originated in the History and Philosophy of Science in an attempt to identify the
epistemological significance of experimental systems, can be extended to cover all
artefacts and activities involving expertise, irrespective of the context of activity. She
feels that the defining hallmark of scientific inquiry - ‘question generating’ and
‘answering providing’ - is no longer restricted to science because there is increasing
evidence of similar patterns of knowledge production in other contexts. Furthermore, she
points out that, in light of today’s software-based artefacts, Rheinberger’s original

31
equation of artefacts with technological instruments is highly problematic. Knorr Cetina
(1997:10) defines software-based artefacts (e.g., e-resources) as:

simultaneously   things­to­be­used   and   things­in­a–process­of­transformation:   (since, 

added   emphasises)   they   undergo   continual   process   of   development   and 
transformation…. These objects are both present (ready­to­be­used) and absent (subject 
to further research) 

She argues that artefacts of this kind can be included in the category of epistemic things
for the following reasons. They are subject to further development, they do not
presuppose a specific pattern of use and they can be used to facilitate a more iterative
relationship between question generating and answering providing activities3.
Knorr Cetina, however, introduces a note of caution into her general argument about the
emergence of epistemic cultures more widely throughout society. She acknowledges that
these cultures are not emerging simply because they are a natural and progressive
development of the application of knowledge within the economy. Knowledge cultures
imply:

an   ‘epistementality’   of   particular   beliefs   about,   for   example,   the   correct   distribution   of 
knowledge,   the   naturalness   of   access   to   it,   the   particular   ways   knowledge   should   be 
handled and inserted into personal and organisational life. Such epistementalities also take 
form   as   particular   organisational   arrangements   or   roles   and   agencies   (Knorr   Cetina 
forthcoming).

This suggests that knowledge cultures have the potential to impact on political, economic
and social life in radically different ways. From her standpoint, since e-resources can be
deployed in a variety of ways and, moreover, depending on the mode of deployment
embed and restrict or enhance inquiry, organisations have to address a new challenge.
They have to re-think organisational beliefs about who will contribute to the process of
determining how artefacts can be used to transform working and learning, how
knowledge about this process of transformation is to be shared and so forth. A
development that, as was noted earlier, presupposes a more inclusive approach as regards
participation of workers in the production, circulation and application of knowledge.
One   of   Knorr   Cetina’s   major   achievements   has   been   to   reconceptualise   our 
understanding of the emergence of and the implications of knowledge cultures 
in   the   social   sciences.   Her   analysis,   however,   stops   short   of   explaining   how 
human mediate external symbolic and cultural systems and artefacts. For this 
reason,   it   is   helpful   to   relate   her   insights   about   culture   and   artefacts   to   the 
concern in Activity Theory, a body of theory that has specifically addressed the 
relationship between activity, learning and knowledge production. 

The role of activity within epistemic cultures.

One of the central premises in AT, a premise that is an implicit feature of Knorr Cetina’s
work, is that the subject-object relation has been transformed through insertion of
3 Knorr Cetina’s observations about the epistemic possibilities of ICT have some affinities with Zuboff’s
(1988) distinction between the ‘informating’ possibilities of ICT. The chief difference between the two
writers appears to be that whereas informating refers to the capaacity of ICT to provide access to
information, epistemic things refers to the more iterative relationship between question generating and
answering activities.

32
artefacts. This means that the relationship between human beings and the external world
is no longer predicated on either the idea of cause and effect or stimulus and response
(Bakhurst 1991). Instead the subject-object relation is ‘mediated’ by artefacts and this
enables humans to come to terms with the world, inscribe value in the world and,
ultimately, to transform it.
The cornerstone of the concept of mediation is the idea that individuals are active agents
in their own psychological, cultural and social development (Cole 1996). At first sight,
this can sometimes appear to be a rather commonplace statement; yet it provides, as Cole
(1996) argues, the underpinning for the following assumptions that the concept of
mediation rests upon. The first assumption is that development is not biologically
pregiven; it occurs as a result of a reciprocal immersion in and engagement with the
social, historical and social context, especially the artefacts, for example, concepts, ideas,
schema and computers, available at a particular period in time. The second assumption is
that human development is shaped and influenced by the practical activities in which
human beings are engaged. The third assumption is that artefacts are both material and
ideal; they are produced by human beings and the very act of production leads humans to
inscribe significance and value into the design/construction of artefacts.
These observations about artefacts constitute an important link between Activity Theory
and Knorr Cetina’s ideas about epistemic cultures. Instead of viewing artefacts as an off-
shoot of culture, Cole, along with Knorr Cetina, view them as the fundamental
constituent of culture; they are created by human beings and used by humans to help
them to coordinate their relationship with the world and with one another. This implies
that artefacts do not exist in isolation as elements of culture: the view that predominates
currently in much of the policy and research literature about e-learning. On the one hand,
artefacts can not only be conceived in terms of a hetarchy of levels that includes cultural
models, for example, question generating/answer providing models, but also in terms of
their materiality and ideality. This is an important distinction because artefacts’
materiality and ideality afford different possibilities for their future development or
application. It is possibility that according to Knorr Cetina, has enabled e-resources to
become epistemic objects. On the other hand, artefacts exist as such only in relation to an
activity, the activity system of cultural context in which they are used. Thus, epistemic
possibilities are realised either in relation to the way in which they are currently utilised
or in relation to yet to be imagined ways of use.
From this perspective, all human activities are mediated through contact with artefacts
and, as a result, have ‘mutlidirectonal consequences’ because activity modifies
individuals in relation to others as well as the medium they are jointly engaged in (Cole
1996). This implies that it is important to remember that all forms of activity are rooted in
culture or ore precisely in the distinctive form of social practice associated with particular
cultures4
Cole illustrated this issue by drawing on Brofennbrenner’s (1979) ideas about the ecology
of human development, Cole points out that human beings develop cognition through
participating in different activities where they have to use symbolic and material artefacts
and simultaneously engage in semiotic reasoning. Thus, from Cole’s standpoint, in order
to understand how knowledge is produced, it is essential to identify how the cultural
context ‘weaves artefacts and activities together’.

4 The terms ‘activity’ and ‘practice’ are sometimes seen as synomous, especially since writers in Actiivty
Theory and Socio-Cultural Theory who employ these terms share a number of assumptions about the link
between human activity/practice and the development of the mind (Lave 1993). In this paper, I use activity
to refer to generic issues, such as, knowledge production and social practice to refer to the processes that
influence or shape knowledge production.

33
The idea of weaving is a helpful notion when considering the relationship between
epistemic activity and culture for a number of reasons. First, it alerts us to the importance
of identifying those social practices that enable people to use artefacts and to engage in
forms of ‘dialogic inquiry’ (Wells 1999) and, in the process, to foster learning and
development and facilitate the production of new knowledge. Second, it sensitises us to
the need for more inclusive frameworks as regards access to knowledge, the rules for the
application of knowledge and so forth (Knorr Cetina forthcoming). Third, it indicates that
the learning of the individual and the system are intertwined with one another (Engeström
2000).
Taken in combination, the above discussion suggest that it is possible to formulate a
tentative characterisation of the concept of epistemic activity. Epistemic activity implies
engagement in ‘question generating and answering providing social practices, involving
the use of artefacts that facilitate the production of new knowledge.

Conceptualising epistemic activity in organisations

The concept of epistemic activity introduces a new way of conceiving of the knowledge
economy/society debate. In contrast to the prevailing view in much of the policy
literature that the increase presence of technological and informational products of
knowledge, and, as a corollary, the primary educational concern is to use e-resources to
increase access to existing educational resources, it presupposes a rather different spatial
and temporal conception of the relationship between knowledge and economic and social
development.
Spatially, as we have seen, the concept presupposes the presence of knowledge cultures
that unfold with its articulation in and between contexts. Temporally, it alerts us, on the
one hand, to the longstanding existence of knowledge work in industrial society, albeit in
the field of science since the eighteenth century. On the other hand, it also alerts us to the
existence of a more pluralistic conception of knowledge work; a conception based on the
emergence of new combinations of theoretical and everyday knowledge sometimes
referred to as ‘polycontextual’ (Engeström et al 1995) or ‘transdisciplinary knowledge
(Gibbons et al 1994) in modern work environments. The emergence of these new forms
of knowledge presuppose that one of the new challenges in modern societies is to identify
the distinctive form of social organization, social practices and artefacts, which enable
people to learn to produce, circulate and use knowledge in workplaces.
One of the most interesting attempts to conceptualise the evolving role of knowledge in
economic activity has been advance by Victor and Boynton (1998). They present the
relationship between economic activity, e-resources and organizational learning in terms
of four knowledge-based responses that orgnaisations can make to the series of
accumulating tensions, which manifest themselves in their production systems. Victor
and Boynton define these responses as the shift from ‘craft’ to ‘mass production’ to
‘process enhancement’ to mass customisation’; they stress, however, that these different
forms of production do not automatically emerge from one another. The transition from
one mode of production to the next only occurs as enterprises respond to specific
possibilities for development that surface in the existing mode of production.
Their starting point is a modern conception of ‘craft work’, which they define as, the
‘invention and creation of high-priced and novel products’. They typify the form of
knowledge that is central to economic activity as ‘tacit’ knowledge (i.e. residing with
workers) and the deployment of e-resources as ‘customised’ design in this system of
working. According to Victor and Boynton, the progress from craft to mass production
occurs at the point when enterprises begin to inquire how to leverage tacit knowledge,

34
that is, to capture and codify it to enhance the volume of production or to extend the
reach of customized systems to meet increased demand from customers.
In contrast to craft work, systems of mass production have the following characteristics.
They rely on the application of ‘articulated knowledge’ (i.e. highly codifed and
procedural) and the use of pre-specified and automated e-resources to ensure the efficient
and effective delivery of products and services. For Victor and Boynton, the paradox of
mass production is that in following instructions without introducing any significant
modifications, enterprises accumulate a new type of knowledge. They define this as
‘practical knowledge’ (i.e. knowing where the instructions are effective and where they
are not). Capturing this practical knowledge to gain market advantage introduces process
improvements; this development however involves enterprises choosing to compete on
product and service differentiation rather than low costs. Furthermore, this strategic
choice also presupposes the adoption of a more inclusive approach towards their workers
than has normally been the case with mass production. It is not inevitable that all
enterprises will chose to respond to these challenges, some will prefer to remain wedded
firmly to mass production.
One of the main challenges of process enhancement is to integrate production and
innovation through the creation of work teams and workplace cultures that foster
intensive and reciprocal communication flows within teams, to facilitate constant
transformation in product and service delivery. Another challenge is to deploy e-
resources to facilitate cross-functional flows of information to support micro-
transformations of products and services, normally within a single organizational context.
Although they employ different terminology, Victor and Boynton argue that process
enhancement presupposes that management link different sections of the workforce, for
examole, procurement, production, distribution, marketing in question generating and
answer providing activities to enhance work flow.
The final shift enterprises can make has come about as customers move beyond quality
and seek products and services that offer exactly what they want. This pressure on
enterprises can, in theory, result in a form of production that Victor and Boynton (1988)
chartacterise as ‘mass customisation’. Rather than the sequenced line of mass production
or the continuous improvement of process enhancement, ‘work is now organized as a
complex, reconfigurable product and service system’.
The transition to such a system is possible because process enhancement tends to
generate a body of ‘architectural knowledge’ about process activities throughout an
enterprise. This form of knowledge reveals to an enterprise the structures of its work
processes, their interconnections, and the possibilities for building them either on each
other or onto customers’ work processes to achieve new combinations or sequences.
Achieving these goals involves a radical extension of enterprises’ conception of e-
resources: they have to be deployed to integrate constantly changing network information
processing and communication requirements to service the production of short-lived
applications, unpredictable product/service requirements and to facilitate a new form of
innovation. According to Victor and Boynton, this can be referred to as ‘co-
configuration’, its hallmark is the integration of customers into enterprises’ business
systems.
A number of conclusions can be drawn from the proceeding analysis of the relationship
between knowledge, e-resources and organizational learning. The first conclusion is that
it is important to acknowledge that the epistemic activity that spurs the movement from
one mode of production to the next results in the production of different forms of
knowledge and involves different forms of learning. This suggests that it may be valuable
to identify a framework for analyzing the range of manifestations of epistemic activity

35
enterprises might be engaged in. The second conclusion is that, in developing such a
framework, it is important to avoid adopting am ‘anthropological’ conception of learning
(Young 1999). In other words treating the relationship between culture and activity as
though it were a naturally recurring process. This implies the need to differentiate more
carefully between the purposes and outcomes of learning. The third conclusion is that it is
important to identify the way in which the ideality of e-resources predisposes them to
serve different purposes, in which case, it may be helpful to differentiate between such
artifacts.

Conceptualising learning, artefacts and epistemic

activity
One of the most promising suggestions for differentiating between the purpose and
outcome of learning has been provided by Engestrom (1994). He has developed a
typology which distinguishes three types of learning - ‘adaptive’, ‘investigative’ and
‘reflexive5’ - and three contexts for learning - ‘discovery’, investigation’ and ‘critique’ -
in terms of the different relationships between learner and context that are presupposed
by each type of learning. His ideas have a number of uses for any attempt to clarify the
contribution that e-resources might make to knowledge creation and the embedding of
epistemic cultures in enterprises.
First, they represent a continuum reflecting the extent to which learning is related to but
separate from discovery. There is no discovery of new knowledge produced in ‘adaptive’
learning because it takes contexts as given and unchangeable and the intention is to come
to terms with the existing stock of knowledge. This is not to dismiss adaptive learning; it
is the social bedrock of the other types of learning and formed the basis for how the
highly sophisticated cultures of pre-industrial societies developed and were reproduced
from generation to generation. Second, ‘investigative’ learning, the desire to explore the
implications of an issue or topic, can generate discoveries as long as opportunities are
provided for new forms of participation in accordance with the principle of question
generating and answer providing within existing ‘communities of practice’. Such
opportunities are likely to generate discoveries that are bounded by the parameters of the
organisation in which they are produced, not unlike the conditions of Kuhn’s ‘normal
science’. ‘Reflexive’ learning, the third category in the typology, seeks to go beyond the
immediate context in which a learner is located. This form of learning tends to emerge
only takes place when investigative learning is unable to deal with the problems
experienced by those in a particular, and their is a felt need to for the context within
which the learning is located to be challenged.
For Engestrom, the starting point for learning is the contradictions that exist within
activity generally (and workplaces, in particular). Crisis points arise when ‘communities
of practice’ confront contradictions that are not immediately resolvable within the
contexts in which they have arisen and require the transformation of existing
‘communities of practice’. This idea has some affinities with Victor and Boynton’s
argument about the pressure points that spur enterprises to chose to take advantage of the
knowledge that has built up inside them. In both cases, for this to happen two conditions
have to be met. First, it must be possible for members of a ‘community of practice’ to call
5 I use the term reflexive learning instead of ‘expanded learning’. Engestrom’s concept of expanded
learning is inextricably bound up with a particular pedagogic approach , which her refers to as, the
‘expansive cycle of learning’. It is my contention that the principles that underpin the concept of expanded
learning have a broader application than is acknowledged by Engestrom, for this reason, I have preferred to
call this type of learning ‘reflexive’ (Guile and Young 1999).

36
into question the context of learning- in other words the ‘community of practice’ itself.
Second, new artefacts, for example theories, ideas, schemas etc need to be available to
support reflexive learning and act as ‘mediating resources’ to help postulate new
relationships between the context and practice.
The second issue raises the question as to how to categorise artefacts; an issue that has
generated a considerable literature on its own (Wartofsky). Since the focus of this paper
is e-resources that can be used to facilitate learning, I have restricted my self to a brief
discussion of those artefacts. As was noted earlier, artefacts are both ideal and material.
Put simply, artefacts are produced by human being and the very act of production leads
humans to inscribe significance into them. Hence artefacts so not exist in isolation as
elements of culture/.
On the most helpful attempts to categories e-resources has been provided by Bates
(1995). He has distinguished between Computer Mediated Resources (i.e. CD Roms/web
sites) and Computer Mediated Communication (i.e. audio, textual/video-conferencing).
The former are primarily concerned with enhancing the presentation of learning material
since they offer access to simulations, provide opportunities for problem solving, they
also can be used to present feedback on performance. The latter offer quite different and
distinctive features. The can facilitate continuous communication and interaction
unencumbered by spatial or temporal considerations. Consequently, they support
dialogue between members of local or distributed communities of practice as much as
between potential members of communities of practice or the possibility of setting up
new communities.
It is the inscription of human values into the design of artefacts that results in Computer
Mediated Resources and Computer Mediated Communication having different design
features and, moreover, having different pedagogic implications. Most commercially
available multimedia programmes and many web sites are characterised by a
pedagogically determined learning process, since instructional objectives have been built
into software programmes. These instructional objectives often restrict users to
responding within pre-determined limits. In contrast, Computer Mediated
Communication provides a more flexible ranger of resources, which can be used to
support collaborative activities and self-directed learning. Therefore, they allow, in
theory, participants to shape, through negotiation and collaboration the goals of working
and learning.

Towards a framework for analysing the relationship

between learning, e-resources and knowledge
creation
The preceding discussion of learning and artefacts has highlighted a number of crucial
issues to be taken into account if we are to understand the contribution that e-resources
make to epistemic activity. For the sake of brevity, they can be summarised as follows.

37
 Diagram 1. Conceptualising learning, ICT and epistemic
activity.

Types of  Types of ICT Epistemic outcomes for 

learning SMEs and Individuals
Computer­accessed  Computer­mediated 
resources communication
(CD Rom, on­line Library,  (Email, file exchange, video 
Websites etc) conferencing etc)
Adaptive  e.g. information retrieval e.g.   acquire   information  little   change   in   working 
learning. from expert(s) and learning practice

e.g. use   information   to  e.g.   use   information   to  ‘additive’ conception of e­

accomplish set task(s) accomplish set task(s) resources
Collaborative   e.g. information retrieval  e.g.   establish   community   of  reconfigured working/
learning. practice learning practices

e.g. discuss information in  e.g.   agree   questions   for  network   e­resources   for 

community of practice exploration/conduct  specific   community   of 
dialogic inquiry practice
e.g.   use   information   to 
address agreed problems e.g.   develop   ideas   to  opportunities to:
reconfigure   work/  ­ participate   in 
learning practices  discussions 
- solve problems
- develop
knowledge/skill
Reflexive  e.g. information retrieval e.g.   establish   internal   and  transformed working/
learning. external   community   of  learning practice 
practice
e.g. identify  network   e­resources   to 
contradictions  e.g.   agree   question   for  expanded   community   of 
in   information/   problem  exploration/dialogic inquiry practice
at hand
.g.   identify   contradictions  opportunities to:
with   community   of  - resituate 
e.g.   use   information   to  practice knowledge/skill
introduce   new   working 
- relate   knowledge/skill 
and learning practices e.g.   model   solutions/ 
to
identify obstacles
- other   communities   of 
e.g.   agree   strategies   to  practice
transform   working/learning  - further   transform 
practice working/learning 
practice

There are different types of learning and each type presupposes a different type of
outcome. There are different types of e-resources and each type presupposes a rather
different from of communication and interaction. Taken in combination, these two
observations point towards a more complex set of issues as regards the use of e-resources
in general, let alone, their contribution to epistemic activity. It is the contention of the
paper that it is possible to use the analytical distinctions raised in the preceding
discussion of learning and e-resources as criterai to formulate a typology for analysing
the relationship between learning, artefacts and knowledge production.

38
Conclusions
The first conclusion which can be drawn from the typology is that in order to use e-
resources to support epistemic activity, it is necessary to begin to create inclusive cultures
inside SMEs that provides increasing opportunities for workers to:
• reflect on experiences and to try our new ideas in practice;
• participate in and discern the rules and protocols of knowledge
producing communities;
• apply new knowledge to develop working and learning practices.
The second conclusion is that establishing new modes of working and learning involves
workers in ‘horizontal’ and ‘vertical’ forms of re-situation (Guile and Young,
forthcoming). The former occurs when individuals carry out a known activity in a new
context. The latter can lead individuals to develop new goals, new action and new
strategies in order to grasp the connections between different activities. This process may
take two forms. First, new patterns of activity emerge from the original context which
constitute a modification of the original activity rather than an alternative realisation of
that activity. Second, it may only be possible to resolve the original problem through
contact with artefacts that lie outside the immediate context.
Reconfiguring or transforming working and learning may involve exploring the value of
concepts and ideas that are external to the enterprise, even though they may have been
developed and debated over a period of time in relation to many similar practical
problem. This involves learners mediating between theoretical knowledge and everyday
knowledge in an attempt to interpret new situations in workplaces in light of the new
knowledge they are developing as well as to deal with counter interpretations.
The third conclusion is that typology of e-resources and epistemic activity is applicable
for analysing similar forms of learning and knowledge production because the concept of
epistemic activity is not restricted spatially or temporally. Rather it reflects ‘places’
within enterprises and the extent to which the practice of ‘question generating’ and
‘answer providing’ are actively encouraged or discouraged. Nevertheless, a caveat has to
be introduced: these implications have to be set in the context of the stage of
development (i.e. Victor and Boynton’s spectrum of development) o epistemic activity
within specific SMEs. As ever, context is critical to practice.

References
Bates, A.W. (1993) Distance Learning and Technology SRHE/OU, Buckingham, UK
Bell
Brofenbrenn
Castells, M. (1996) The Rise of the Network Society, Vol 1. MacMillan, London
Cole, M. (1996) Cultural Psychology Cambridge University Press, Cambridge
Daniels. H. (2001) Vygotsky and Pedagogy Routledge, London
Engeström, Y (1993) Training for Change, ILO, Geneva, Switzerland
Engeström, Y. Engeström, R., and Karkkainen, M. (1995) ‘Polycontextuality and
boundary crossing in expert cognition: Learning and problem solving in complex work
activities’ Learning and Instruction 5. 1. pp. 319-336
Engeström, Y. (2000) Expansive Learning at Work: Toward an activity-theoretical
reconceptualisation In: Journal of Education and Work, 14, 1, pp.

39
Gibbons, M. et al (1994) The New Production of Knowledge Sage, London
Guile, D. From ‘credentialism’ to the ‘practice of learning’: rethinking learning for the
knowledge society, Policy Futures in Education, 1, 1.
Knorr Cetina, K. Epistemic Communities, Harvard Education Press, Harvard 1999
Knorr Cetina, K. (forthcoming) Five Transitions Towards a Knowledge Society
Lave, J. (1993) The Practice of Learning In Chaicklin, S. and Lave, J. (eds)
Understanding Practice Cambridge University Press, Cambridge
Rheinberger, J. (1992) Experiment, Difference and Writing in Studies in the History and
Philosophy of Science Vol 23, No 2. 305-21
Victor, B. and Boynton, A. Invented Here Harvard Business School Press, Harvard
Wells, G. (1999) Dialogic Inquiry, New York: Cambridge University Press.

40
41
Localization of software and learning material for
SMEs:
how is it possible?

Bernard Blandin, CESI (France)

Introduction
This paper revisits research undertaken ten years ago through the COMETT programme
(Auvinen, Blandin & al., 1994) aiming to identify the cultural differences which have an
impact on learning in order to facilitate the localization and adaptation of learning
material across boundaries. This approach appears to be still valid, but should be
deepened, since further research done in the meantime and sociological concepts such as
the “social world” (Blandin, 2002aand “learning culture” (Blandin, 2003) are likely to
help refine the approach.
It is not enough to adapt content: the “form” of the learning material has also to be
localized. Designing effective learning environments has to take into account social and
cultural factors, but also learning software and learning material usability has to be
considered as “situated” (Blandin, 2003). Occasions and conditions for the use of
learning material arise directly out of the context of learning activities which are
implemented.
As a result of these findings, it appears that the localization process as well as the design
process for software and/or learning material cannot be isolated from the design or the
adaptation of learning situations in which the software or the learning material is to be
used.
In this paper I will first present background research on cultural aspects of learning and
their particular relevance to SMEs, then I will examine how my standpoint on “situated
usability” of software and learning material impact on these results and finally, I will
propose a two-tier development process as a condition for the successful design and/or
localization of software and learning materials.

Culture, Learning and SMEs

It is not common in discussions of ‘Learning’ to look at anthropology or sociology. This
is why, in my opinion, cultural aspects of learning are not sufficiently explored.
Differences in national cultures and national value systems and their impact in the field of
management has been discussed by several authors, including Hofstede (1991),
Hampden-Turner and Trompenaars (1993).
Hofstede identified the following areas which impact on the managerial relationship:
- Social inequality, including the relationship with authority,
- The relationship between the individual and the group,
- Concepts of masculinity and femininity and their social implications,
- Ways of dealing with uncertainty, relating to the control of aggression and the
expression of emotions,
- Time orientation (short-term or long-term).

42
Hampden-Turner and Trompenaars identified the following ‘dilemmas’ which might
create tension when national value systems are in confrontation:
- Universalism / particularism,
- Analysing / integrating,
- Individualism / communitarianism,
- Inner-directed orientation / outer-directed orientation,
- Time as sequence / time as synchronisation,
- Achieved status / ascribed status,
- Equality / hierarchy.
In the field of education and training, our “Cultural Impact on Learning” research pointed
out obvious differences from one country to another in terms of structure, administration,
curricula, regulations, etc. at every level6. But, beyond the cultural factors, there are also
individual factors which impact on the way people learn and their learning strategies,
such as what psychologists call ‘Learning Styles’. Different researchers have identified
several components of learning styles. The most advanced work comes from Canada,
with the work of Hill and his follower, La Montagne. The components of learning styles,
which partly overlap cultural factors, are the following (La Montagne, 1985):
- Learning support (relationship with authority, with the group, with the
individuals),
- Decoding of information (practitioner / theorist),
- Processing of information (inductive / deductive).
Another author, E.T. Hall, explored the cultural differences related to the organisation
and the use of space which impact on communication and interpersonal relationships
(1966). He identified three “spheres” around a person, the dimensions of which differs
according to culture. These were:
- Intimate space: the closest ‘bubble’ of space surrounding a person. Entry into
this space is allowable to only the closest friends and intimates.
- Social and consultative spaces: the spaces in which people feel comfortable
conducting routine social interactions with acquaintances as well as strangers.
- Public space: the area of space beyond which people will perceive interactions
as impersonal and relatively anonymous.
Hall developed his theory of “proxemics”, arguing that human perceptions of space,
although derived from sensory apparatus that all humans share, are moulded and
patterned by culture.
There is no need to demonstrate that landscape, environment, and human settlements vary
according to location, climate and other geographic parameters. Scenery and the
environment to which people are used also constitute cultural references, which label
implicitly the background of any scene as “familiar”, “from elsewhere”, or “exotic”.
Based on this initial research, and on eight case studies, we proposed, as an output of the
“Cultural Impact on Learning” research, a set of recommendations concerning the

6 More recent reports have added to this work. See, for example, the European Commission Study Group on
Education and Training report ‘Accomplishing Europe through Education and Training’, and in particular
the Annex (Part VI).

43
adaptation of the content of learning material across borders. The main findings were as
follows.
“Do not only translate, but take into consideration, and be prepared to modify the
following:
- Scenic elements and lifestyles,
- Economic and institutional systems,
- Social relations (authority, gender, interpersonal contact etc.),
- Language (categories, structures etc.),
- Value systems and beliefs,
- Learning styles and learning strategies.”
Learning material had to be adapted for use in any particular country, to take account of
national cultures. I will now demonstrate that even in a given country, there is no single
culture. W have already seen, with learning styles, that we might be obliged to take into
account more individual characteristics than culture.
E.T. Hall, in some way, expressed the same idea as Marcel Mauss (1950) with his
concept of “body techniques”, arguing that the way you walk, the way you sit, the way
you swim etc. are cultural, not natural, and that they are learned. Furthermore, recent
approaches in linguistics make the assumption that our categories (Lakoff, 1987) and our
ways of thinking, though framed by common schemata based on body structure, are also
moulded and patterned by culture and language (Lakoff & Johnson, 1999). This
reinforces the hypothesis by Sapir-Whorf that conceptual models are formatted by
language and categories, and are specific to a particular population. Lakoff provides
numerous examples, including the Dyirbal (an Australian aboriginal tribe) classification
of the universe into 4 categories, a classification lost when younger generations began to
learn English and the Japanese ‘Hon’ category, of regrouping long objects (hairs as well
as sticks or pencils, candles, etc.) which has no equivalent in any other language (Lakoff,
1987).
Even what could appear to us as a basic human categorisation, based on human
physiology, such as colour, is a cultural product, and the ability to distinguish between
two colours like blue and green appears to be related to the fact that these categories can
be named in a given language, as was illustrated by Kay and Kempton’s experiment
comparing English speakers and Dani-only speaking speakers. The Dani are a New
Guinea tribe whose language distinguishes own two colours and for whom blue and
green are the same (Kay & Kempton, 1984). The epistemological consequences of such
an experiment are analysed by Varela, Thompson & Rosch (1992). I have demonstrated
that objects also contribute to these moulding and patterning of our minds, since the use
of objects requires “social schemata of uses”, which are themselves cultural and vary
from one country to another (Blandin, 2002b). Finally, it is now becoming evident that
almost every behavioural or communication skill or thinking capacity is shaped by our
culture and through our language.
I will also argue that the word ‘culture’ should not implicitly be taken as synonymous to
‘national culture’. It can also refer to smaller groups: one of the first sociologist to use the
expression “social world” to describe the culture of a small group was Howard S. Becker
in his famous study on outsiders (Becker, 1963). This notion was used in interactionist
sociology, and re-invented by the French sociologists Boltanski & Thevenot, who tried to

44
understand the judgement process7, and the existence of different “value measurement
systems” through which groups of people evaluate the world in which they live (persons,
objects, event etc.). These “measurement systems”, determine different “Cities” (“Cités”,
in French) or “social worlds” within French national culture (Boltanski & Thévenot,
1991). These “Cities” are rooted in the history of social bodies and in their value systems:
- the “inspired” world (or the world of artists),
- the “domestic” world (or the world of “we-relationships”),
- the world of “fame”,
- the “civic” world,
- the world of “merchants”,
- the “industrial” world.
For Boltanski & Thevenot, the consequence of these sub-cultures is that social relations
between people from different “worlds” take the form of conflicts or compromises.
Using Boltanski & Thevenot’s approach, it is possible to demonstrate that there are more
“social worlds” than the six they described. In particular, SMEs founders, for me, belongs
to a specific “social world”. SMEs are generally analysed from an economic standpoint.
But some research has shown that SMEs founders share common sociological
characteristics, defined by the term “entrepreneurship”. Most of the research
distinguishes between the “entrepreneur” and the manager of a large company:
entrepreneurship is defined as “the pursuit of a discontinuous opportunity”(Carlton,
Hofer & Meeks, 1998), whereas management is the maintenance of the continuity of an
organisation. For these authors, “management” starts when “entrepreneurship” finishes,
when the organisation reaches self-sustainability. According to them, this means
entrepreneurship, or establishing a SME, is a specific behaviour.
The traits most frequently cited as characterising entrepreneurs include the desire for
independence, locus of control, creativity, a risk taking propensity and the need for
achievement (Carlton, Hofer & Meeks, 1998). French research (Aumont & Mesnier,
1992) presents similar results: entrepreneurship is the achievement of a project and the
entrepreneur is characterized by his or her need to face challenges, the desire for
independence, a risk taking propensity, the need for action and creation, the need for
achievement and the ability to anticipate and to seize opportunities. If we consider
“Entrepreneurship” as the “Common Principle”, and the “Entrepreneur” as the “Grand
State”, “Independence” as “Personal dignity”, “Success” as the main “Judgement
criterion” etc., “Entrepreneurship” can clearly be seen as a particular “City” or a
particular “social world”, as meant by Boltanski & Thevenot. It means that SMEs, in a
given country, have their own sub-culture, rooted in the “Entrepreneurial City”, which is
different from the “Industrial world” of the managers of big firms, or the “Merchant
world” of capitalists and shareholders.
My hypothesis is that this particular culture of SME owners frames both their own
learning strategies and their training policies as employers. This raises a number of
paradoxes. SMEs are generally considered as lacking a training culture 8, and all statistics
show that SME employees participation in training is low compared with larger
organisations. On the other hand, “undertaking”, which is the core of entrepreneurship, is
7 Their question was: How can people evaluate things and share their evaluation; how is it possible to have
a common appreciation of things?
8 See the Cedra ICT Research Network report (Attwell, forthcoming), or, for France, the survey on training

policies in very small enterprises (Bentabet, Michun & Trouve,1999).

45
considered, together with “researching”, a process which generates and supports the
learning process and the desire to learn (Aumont & Mesnier, 1992). Self-directed
learning requires some of the entrepreneur behavioural characteristics: autonomy, a
feeling of responsibility, self-management and self-monitoring of learning process and
learning paths, viewing problems as challenges (Mardzia Hayati, 2001). And
entrepreneurs, when asked about their learning strategies, appear to be, in the majority,
self-directed learners or self-educated learners, or to use another term, “autodidacts” (Le
Meur, 1998).
Duplaa’s recent research in France (2002) reinforces these findings. First, he confirms
what was pointed out in the Cedra ICT Research Network report (Attwell, forthcoming):
the attitude of SMEs “managers” emerged as the single most decisive factor in
influencing training policies. This in turn implies managers have to be involved in all
decisions about training and about the design and localisation of. Duplaa (2002) suggests
the need for alternative solutions to formal training within SMEs including associations
or clubs to share experience and good practices between entrepreneurs and coaching and
on-the-job training for employees. In fact, informal learning situations appear to be the
rule in the “Entrepreneurial City”. People learn in SMEs, but not within formal training
systems: some entrepreneurs interviewed by Aumont & Mesnier (1992) even say that
they learn all the time. A common learning style, described by Bentabet, Michun &
Trouve (1999), has the following characteristics:

- Learning support: the owner of the company is considered as the one

who knows. Interpersonal relations are very important since value
systems are transmitted together with knowledge and know-how.
- Decoding of information is by the practitioner. Practical solutions are
required, theory is considered as not useful or far-fetched.
- Processing of information is inductive. Working situations and problems
encountered must be the starting point.
At this stage, it appears that SMEs belong to a particular social world, the
“Entrepreneurial City”, with a particular “learning culture” (Brown, Collins & Duguid,
19899) generally associated with a specific learning style. This might explain the
difficulties in transfering learning material and training models from training institutions
or large companies to SMEs. Local adaptation of the content of learning material, as
proposed in the research on the “Cultural Impact on Learning”, might not be enough.
Before discussing this issue, we need to explore another aspect of the use of learning
material and learning software relating to its “form”, which is generally embedded within
theories of design as “Usability”.

Situated Usability
This section is based on my chapter in a book dedicated to the evaluation of the usability
of online learning (Blandin, 2003). In this paper I will only provide an outline of the
ideas. I will define the word “usability” and then summarize why usability has to be
considered as situated, and finally examine the conditions which might facilitate the use
of learning material as self-study material.
9 Brown, Collins & Duguid (1989) consider the differences between learning process as implemented in
school and in activity-based learning processes as naturally developed by practitioners in their professional
life or “Just Plain Folks” in their daily life as a consequence of two different “Cultures of Learning”.

46
In its broad sense, “usability” addresses the relationship between tools and their users.
Usability depends on a number of factors measuring how well the functionality of the
tool fits user needs. For software, this may include how well the flow through the
application fits user tasks, how well the response of the application fits user expectations.
Generally, this relationship between a human being and an artefact or an object is
considered as independent of any contextual, social or cultural aspects: usability criteria
relate to “Human Factors” considered as universal.
From a designer’s viewpoint, usability is seen as a relationship between a human being
and an artefact which measures the productivity of a user using the artefact (Nielsen,
1994). This may appear confusing, since it does not make a distinction between two
usability-related concepts, “ease of use” and “usefulness”. Therefore, I prefer to use the
extended notion of usability proposed by Notess, following Norman and the “User-
Centred Design” School of University of California, which distinguishes clearly between
the two usability-related concepts, but takes tboth into account into its notion of
“usability” (Notess, 2001). This also answers the questions of the differenced between
“Utility” and “Usability” raised by authors like Grudin (1992) or more recently Tricot &
Tricot (2000) and Tricot & Lafontaine (2002).
Donald Norman (1988) proposes the following basic principles for good design for
usability:
Rule 1: Provide a good conceptual model,
Rule 2: Make things visible,
Rule 3: Map the controls, their movements and their results in the real world,
Rule 4: Provide feed-back for any actions.
Norman’s rules and ser-centred design principles, named “Usability Engineering” by
Notess (2001), apply to all software including learning software. Elaboration on these
rules led Nielsen, the father of software usability, to propose his “Ten Usability
Heuristics” (Nielsen, undated), in which we find the same principles. Usability, when
applied to software design and engineering, is synonymous with “ergonomic issues”. It
attempts to define rules to design the application in order to match users behaviour to
various types of equipment and different languages. For example, Nielsen, in his book on
web usability, focuses on page, content, site and intranet design, addressing issues such
as viewing pages on various monitor sizes, as opposed to writing concise texts for
“scanability”; or paying attention to users with disabilities or international users (1999).
Quinn (2001) describes an example of widespread usability problems concerning learning
software:
- Counter-intuitive reading order of on-screen material” (breaking Rule 1),
- Failure to relate to the real world experience of the user” (breaking Rule 3),
- Poor presentation of key information” (breaking Rule 2),
- Lack of accessibility, even in most basic sense” (breaking simultaneously all
the rules, as shown in the examples given in the following pages of Quinn’s
paper).
I would argue that to apply these types of rules to designing learning software would
never guarantee that they would be “usable”. I will, in the following section, make the
case for a strong dependence of “usability” on social and cultural aspects because using
an object is a social activity, implicitly implying social relations belonging to different
registers (Blandin, 2002b). Universal “Human Factors” on which to build usability
criteria do not exist but, instead, we have to take into account contingency factors, for

47
which we need appropriate conceptualisation. My arguments, which are summarized
here, take into account both cognitive and sociological standpoints (Blandin, 2003).
The main cognitive standpoints are based on the research presented in the first section,
which leads to the conclusion that since our conceptual models are cultural in essence,
there is no good conceptual model. This means that the usability of a software tool, even
in a restricted sense, has to be related to the cultural context in which the tool has to be
used. As an example, the “intuitive reading order” rule proposed by Quinn (2001) cannot
produce the same screen layout in Western Countries, in Japan, or in Arabic Countries,
simply because reading order in these cultures is different!
The second point I discussed is the notion of “real world”, which also appeared in the
second of Nielsen’s Usability Heuristics (undated). From a social constructivism
viewpoint, the “real world” appears as a social construction which arise from the
situation confronting the subject (Blandin, 2002b). The “Situated Cognition” approach is
rooted in similar considerations: concerning tools, Brown, Collins & Duguid (1989) state
that “the occasions and conditions for use arise directly out of the context of activities of
each community that uses the tool, framed by the way members of that community see the
world. The community and its viewpoint, quite as much as the tool itself, determine how a
tool is used. Thus, carpenters and cabinet makers use chisels differently.” Again, this
leads us to the conclusion that social and cultural context has to be taken into account to
evaluate usability.
Thirdly, I pointed to evidence from the Sociology of Uses. Punie’s survey of “Non Uses”
of several ICTs applications in Flemish Households (1997) reveals that there is a pre-
condition to use a tool: the tool must respond to a particular need. “No need” is the main
reason people say they do not use devices like videotape recorders, handheld computers
or Pay-TV. “Usefulness” appears as a necessary prerequisite to use a tool. “No need”
does not mean no relation at all with the object: there could be some cognitive or
emotional relation, or the object could be considered as an indicator of social status, etc.
But this type of relation does not mean the tool will be used, because it takes place in a
register that is different to the “utility register” (Blandin, 2002b).
If we agree “Usability addresses the relationship between tools and their users”
(Usability First, undated), “usefulness” has to be considered as a component of
“usability” in its broader sense. This is also applies learning software. Recent papers on
e-learning usability (Quinn, 2001; Notess, 2001) have shown e-learning programs are not
used because of users’ lack of motivation. This has to be taken into account in assessing
their usability (Notess, 2001). Technology museums are full of products that people
found easy to use, but which have never been used (Jennings, 2001), and many current e-
learning programs seem to be good examples of such products, according to Quinn
(2001)!
That e-learning programs are not used is also one of the findings of Elliott Masie’s survey
(2001. The survey, based on American companies, shows that the key factors to
successfully implement corporate e-learning – i.e. to have it used by employees – are
determined by the organisation in which the programs are implemented, and not, or very
little, by the intrinsic characteristics of the programs themselves. According to Masie, the
following ‘motivators’ determine the acceptance of e-learning:
- good advertising and championing within the company,
- time and support provided during working hours,
- creation of an e-learning culture within the company,
- provision of incentives such as peer recognition and career advance.

48
The theory of the “Seven Pillars of Self-Directed Learning” proposed by Philippe Carre
(1992) for the successful implementation of Self-Directed Learning within an
organisation – whatever the material – shows that most of the conditions for success are
dependent on organisational decisions (5 pillars out of 7!). This suggests that to
successfully implement e-learning, organisational support for the motivation of learners
plays the main part. Thus, from the viewpoint of the Sociology of Organisations, social
and cultural contexts have to be taken into account to set up criteria for usefulness and
therefore for usability criteria. Again, this highlights the importance of cultural and
sociological context in determining the “usability” of tools.
The existence of “Social Schemata of Uses” associated with the use of tools (Blandin,
2002b) make also the case for encompassing factors describing the environment of uses
in any usability assessment. An example of this is my usability problems in North
America in using domestic electric appliances like hair-dryer or boilers complying to the
“User-Centred Design” recommendations (Norman, 1988). These appliances have a “On”
button and an “Off” button, instead of an “On-Off” switch as we have in Europe. Despite
cognitively knowing this, my fingers simply refuse to use a second button to switch off
the device and my reflex action, always repeated, is to try to press again the “On” button.
Another illustration is the “Accountant case”, I cited in my paper (Blandin, 2003). A
woman was using CBT software to learn algebra. She was systematically making errors
in the exercises provided on the bottom line of the screen, though was successfully
completing similar exercises on different parts of the screen. It turned out She was an
accountant, used to adding a column at the bottom of the column, and this prevalent
schema prevented her from adding numbers in a different way.
This is why I am inclined to propose the notion of “Situated Usability” to name a set of
“heuristics” that account for describing how the environment impacts on usability factors.
These heuristics are the following (Blandin, 2003):
(1)Social Schemata of Uses: to be used within a given community a tool should
embed common Social Schemata of Uses of this community;
(2)Type of Action: to be used in a given situation a tool should correspond to
users needs and purposes in this situation, and allow the performance of a given
action;
(3)Culture of the Users: to be used within a given community a tool should
convey representations and practices which are considered as “common sense
knowledge” by the user;
(4)Culture of the Environment: to be used within a given community, a tool
should convey representations and practices which are considered as “common
sense knowledge” by the community;
(5)Tool Efficiency in a given situation: to be used in a given situation a tool
should have proven efficiency in such a situation;
(6)Ability of the User to use the Tool: to be used in a given situation, a tool must
be mastered, to some extent, by the user which also means that an object does not
become a tool immediately or even within a short time;
(7)Motivation of the User to use the Tool: to be used in a given situation, a tool
should interest the user sufficiently to use the tool rather than acting in a different
way.
Users do not live or act in an abstract world alone with the tool they are using. Users, as
human beings, live and act in a world which is at the same time social and material. Use
of tools is deeply rooted in users’ culture, which means that it is rooted in previous
learning from their social and material environment. As a consequence, usability must
take into account the user’s social and material environment. This is why usability has to

49
be “situated”. This simply means that usability has to take into account user experience,
in its broad sense, and this is certainly one of the best Design principles! But it is also
probably the most difficult to implement. “User experience”, and in particular the
relationship between the user and an object or an artefact is context-dependent, and
therefore requires a sociological standpoint to be analysed and fully understood.
If we now return to learning software and learning material, I assume that the
epistemological stance of the teacher or trainer, the motivation of the learner, the
organisational learning culture and environmental factors strongly interact in determining
the conditions for their use. I will conclude this section with a brief outline of the
conditions which facilitate the successful use of learning software or learning material for
adult learning. My hypothesis is that configurations which work should embed what
Malcolm Knowles in his book Self-Directed Learning (Knowles, 1975) and others (Long,
1995) consider the main characteristic of adult learning. In other words, they should
support the development of Self-Directed Learning. The motivators determining the
acceptance of the learning material will be the same as those listed above as the “Seven
Pillars of Self-Directed Learning” (Carre, 1992).
However “Self-Directed Learning” is a concept with no place within learning theories
such as Behaviourism or Instructionalism which deny the learner as the locus of control
and refuse the idea the learner may acquire knowledge or know-how without the help of
someone else who knows. Constructivism appears as the only current theory of learning in
which learner’s activity is considered to play a major part, and therefore it appears as the
only theory compatible with the implementation of Self-Directed Learning. This is why I
assume that there could be a correlation between the effective use of learning material –
in particular learning software – and learner’s motivation and his or her Self-Directed
Learning Readiness (Guglielmino, 1977). When Self-Directed Learning is implemented
within an organisation, then usability of learning software also depends on the following
criteria:
- sufficient support and appropriate to match the learner’s ranking on the Self-
Directed Learning Readiness Scale;
- an appropriate learning culture and support environment to match the expected
motivation of learners.
Several conditions are necessary in the use of a tool in a given environment. These
conditions can be analysed using my “Situated Usability Heuristics”. In the case of
learning software, or more generally of learning material, these heuristics can also be
used with the word “culture” being replaced by “learning culture”. If we consider a
particular “social world”, then the “community” designates this “social world”. The
reader will easily transpose the heuristics in order to determine the conditions to use
effectively learning software in the “Entrepreneurial City”.

Designing and Localizing Learning Material

We are now able to assemble the pieces of the puzzle to answer the question posed in the
title: how is it possible to design and localize learning material for SMEs? This section
describes a general process, and its relation to the “Entrepreneurial City”.
The evaluation of the usability of learning material cannot separate the material from the
situations in which this material is used. This implies that both design and localization
processes have to focus at the same time on the material and the conditions of its use by a
given target group, with its particular cultural characteristics. The distinction which I
made between the content and the form appears to be very useful here, since it helps to
rationalise the design and localization processes through a two-tier model, each level
dealing with specific objects as detailed in the table below (Table 1).

50
The process for each tier follows the five classical phases of instructional design:
1) Needs Analysis,
2) Design,
3) Development,
4) Implementation
5) Evaluation.
But since we have two different domains, which can be processed independently, it is
necessary to cross-check the compatibility between the form and the content, at least
during the design phase, and to take this into account in the evaluation.

Context of use Learning material

Content-related - Skills and knowledge standards - Discipline, level, type of
objects - Objectives and evaluation modes content
- Didactical programme and learning - Learning activities in which
activities it is used
- Learning culture of the actors (learners, - Social, economical,
tutors, trainers etc.) institutional systems
- Learning styles and learning strategies - Social relations, value
(learners) systems and beliefs
- Motivation (learners)

Form-related objects - Pedagogical approach - Type of material (paper-

- Types of learning activities based, audiotape, videotape,
- Rhythm, duration of activities software etc.)
- Learning place - Scenic elements and
- Learning space architecture and lifestyles
organisation - Language
- Technological system
- Support system and communication
modes

Adaptation and localization of learning material is necessary, though not sufficient, and
often both the form and the content of learning material will need adaptation in the
localization process. Thus, when the learning material is software or is digital,
localization will be greatly facilitated if the material is built around a database and if
texts, pictures, sounds, films etc. are considered as parameters and described in the
database. In that case, the content-related or the form-related objects can themselves be
described as metadata to help identify the material10.
Finally, as a conclusion, I must point out some assumptions about learning materials and
how to adapt them to SMEs. From Table 1, we can assume that:
- in terms of content, if the subject is not aforeign language or foreign
culture, the material has to refer to national socio-economical context and
to the “Entrepreneurial City” value system and relational models;
- in terms of form, if the subject is not a foreign language or foreign
culture, the material has to be in national language, and must feature local
scenes and “ordinary” lifestyles.

10Though it is not my purpose here to discuss the issue of metadata standards, it is obvious that the
proposed approach challenges the current models, like AICC, SCORM, or the ISO SC36 proposals
because, even if elements of the context of use can be described in some models, form and content are
never distinguished.

51
There are no other conditions for the material. But, it is also of the utmost importance that
the materials be inserted in the right context of use, which has to take into account the
learning culture and the learning style of the “Entrepreneurial City”, including:
- interpersonal relations between the learner and tutor from the same
world,
- a group of peer learners from the same geographical area, but avoiding
competitors,
- activity-based learning, with a practical inductive approach rooted in
case studies, whenever possible provided by the learners themselves,
- ‘side activities’, including social events, contests etc. to foster
motivation,
- short sequences (1 or 2 hours), which could take place at the beginning
or end of a f working day,
- if travel is needed to the learning place, it should be in the close
neighbourhood (typically not more than a ten minutes trip).
These appear to be common sense recommendations. But they have still to be confronted
with the “real world”, and, in such matters, there is but one question: if we build it, will
they come?

References
Aumont, B. & Mesnier, P.M. (1992). L’acte d’apprendre. PUF, Paris (France).
Auvinen, A.M. Blandin, B. Chapman, P. Dondi, C. Evans, R. Goldstone, L. Sharratt, R.
edited by Johnstone, A. (1994). Cultural impact on learning : A practical guide to
managing the effective adaptation of learning materials across international boundaries.
Contact, Manchester (UK).
Becker, H.S. (1963). Outsiders: Studies in the Sociology of Deviance. The Free Press,
New-York (NJ).
Bentabet, E. Michun, S. & Trouve, P. (1999). Gestion des hommes et formation dans les
très petites entreprises. Etude n°72. CEREQ, Marseille (France).
Blandin, B. (2003). Usability Evaluation of Online Learning Programmes: a Sociological
Standpoint”, in Ghaoui, C. (ed). Usability Evaluation of Online Learning. Idea Group
Inc. (PA), p313-330
Blandin, B. (2002a). Les mondes sociaux de la formation, in Les TIC au service des
nouveaux dispositifs de formation. Education Permanente n° 152, 2002-3, p 199-211.
Blandin, B. (2002b). La construction du social par les objets. PUF, Paris (France).
Brown, J.S. Collins, A. & Duguid, P. (1989). Situated Cognition and the Culture of
Learning, in Educational Researcher Vol.18 n°1, 32-42.
Carre, P. (1992). L’autoformation dans la formation professionnelle. La Documentation
française, Paris (France).
Carton, R.B. Hofer, C.W. & Meeks, M.D. (1998). The Entrepreneur and
Entrepreneurship – Operational Definitions of their Role in Society. Paper presented at
the annual International Council for Small Business conference, Singapore. Retrieved
January 29, 2003: http://www.sbaer.uca.edu/Research/1998/ICSB/k004.htm
Cedra ICT Research Network. (2002). Joint Cedefop-European Commission Study on e-
learningin SMEs. Cedefop, Thessaloniki (Greece).

52
Duplaa, E. (2002). Vers un dispositif FOAD adapté aux toutes petites entreprises. Cesi,
Paris (France).
European Commission Study Group on Education and Training. (1996). Accomplishing
Europe through Education and Training. Retrieved January 29, 2003, at the following
URL: http://europa.eu.int/comm/education/reflex/en/homeen.html
Grudin, J. (1992). Utility and Usability: Research issues and development context, in
Interacting with computers, Vol 4, n°2, 209-217.
Guglielmino, L. (1977). Development of the Self-Directed Learning Readiness Scale.
Doctoral Dissertation, University of Georgia, Athens (Georgia).
Hall, E.T. (1966). The Hidden Dimension. Doubleday, New-York (NJ).
Hampden-Turner, C & Trompenaars, F. (1993). The Seven Cultures of Capitalism.
Currency Doubleday, New-York (NJ).
Hofstede, G. (1991). Culture and Organizations: Software of the Mind. McGraw-Hill,
New-York (NJ).
Jennings, T. (2001). Dead Media Project Working Notes by Categories. Retrieved
August, 25, 2001, from http://www.deadmedia.org/notes/index-cat.html
Kay, P. & Kempton, W. (1984). What is the Sapir-Whorf Hypothesis, in American
Anthropologist Vol. 86, n°1, 65-79.
Knowles, M. (1975). Self-Directed Learning: A Guide for Learners and Teachers.
Association Press, New York (NJ).
Lakoff, G. (1987). Women, Fire and Dangerous Things. What Categories Reveal about
the Mind. The University of Chicago Press, Chicago and London.
Lakoff, G. & Johnson, M. (1999). Philosophy in the Flesh. The Embodied Mind and its
Challenge to Western Thought. Basic Books, New-York (NJ).
La Montagne, C. (1985). Le profil d’apprentissage, vue d’ensemble. IRPA, St-Hubert
(Quebec).
Le Meur, G. (1998). Les nouveaux autodidactes. Chronique sociale, Lyon (France).
Long, H. (ed.) (1995). New dimensions in Self-Directed Learning. University of
Oklahoma, Norman (Okl).
Marzia Hayati, A. (2001). Self-Directed Learning. ERIC Digest. Retrieved January 30,
2003: http://www.ericfacility.net/ericdigests/ed459458.html
Masie, E. (2001). E-Learning: If we Build It, Will They Come ? American Society for
Training and Development, Alexandria (Va).
Mauss, M. (1950). Sociologie et anthropologie. Quadrige / PUF, Paris (France).
Nielsen, J. (1999). Designing Web Usability: the Practice of Simplicity. New Riders
Publishing, Indianapolis (In).
Nielsen, J. (1994). Usability Engineering. Morgan Kaufmann Publishers, San Francisco
(Ca).
Nielsen, J. (no date). Jacob Nielsen Online Writings on Heuristics Evaluation. Retrieved
August, 25, 2001, from http://www.useit.com/papers/heuristic/
Norman, D. (1988). The Psychology of Everyday Things. Basic Books, (USA).
Notess, M. (2001). Usability, User Experience, and Learner Experience. E-Learn
Magazine In-Depth Tutorials. Retrieved August, 25, 2001 from the following URL:
http://www.elearnmag.org/subpage/sub_page.cfm?section=4&list_item=2&page=1

53
Punie, Y. (1997). Imagining “non uses”. Rejection of ICTs in Flemish Households, in
Imagining Uses. Proceedings of the 1st International Conference, May, 27-29,1997.
Bordeaux (France), 165-176.
Quinn, A. (2001). Why people can’t use e-learning. What the e-learning sector needs to
learn about usability. Retrieved August, 25, 2001, from the following URL:
http://infocentre.frontend.com/servlet/Infocentre?
access=no&page=article&rows=5&id=163
Tricot, A. & Tricot, M. (2000). Un cadre formel pour interpréter les liens entre
utilisabilité et utilité des systèmes d’information, in Actes du colloque Ergo-IHM. Biarritz
(France), 195-202.
Tricot, A. & Lafontaine, J. (2002). Evaluer l’utilisation d’un outil multimédia et
l’apprentissage, in Apprentissage des langues et technologies : usages en émergence. Clé
International, Paris (France), 45-56.
Usability First (no date). Introduction to usability. Retrieved August, 25, 2001 from
http://www.usabilityfirst.com/intro/index.txl
Varela F., Thompson E. & Rosch E. (1992). The Embodied Mind. Cognitive Science and
Human Experience. MIT Press. Cambridge (Mass).

54
E-learning Content and Software Localization

George Bekiardis, ERGONKEK (Greece)

Introduction
Few would dispute that the rapid developments in today’s information technology have
revolutionized traditional learning environments. Indeed, the concept of e-learning was
born from the fusion of information and communication technologies and it has mirrored
the achievements of such technology by providing users with ever-greater learning
opportunities. The implementation of e-learning should expand the range and numbers of
skilled workers through vocational training and education. But, besides the technology
itself, the major factors that impact on the successful implementation of vocational e-
learning programmes are content and software.
The majority of e-learning content produced globally is in English. What happened in
non-English speaking countries in Europe? Some European countries (France, Italy,
Greece etc) are proudly insisting on their native languages. In these countries there is less
willingness to use English even for technical subjects. It is obvious that content
localization is becoming ever more necessary in Europe.
The purpose of this paper is to explore the issues that influence the localization of
vocational e-learning content and to propose methods for effective localization of e-
learning.

What is Localization
The best definition for localization is from whatis.com (http://www.whatis.com):
“Localization is the process of adapting a product or service to a particular language,
culture, and desired local ‘look-and-feel’. Ideally, a product or service is developed so
that localization is relatively easy to achieve - for example, by creating technical
illustrations for manuals in which the text can easily be changed to another language and
allowing some expansion room for this purpose. This enabling process is termed
internationalisation. An internationalised product or service is therefore easier to localize.
The process of first enabling a product or service to be localized and then localizing it for
different national audiences is sometimes known as globalisation.
In localizing a product, in addition to idiomatic language translation, such details as time
zones, money, national holidays, local colour sensitivities, product or service names,
gender roles, and geographic examples must all be considered.”
A successfully localized service or product is one that appears to have been developed
within the local culture.

E-learning content and software localization for a

specific country and a specific target group
Before starting the localization of e-learning content and software we must exactly
specify the following:
Target country and market (e.g. Greek vocational training institutes)
Target audience (e.g. trainees and trainers)

55
Use of content and software (e.g. informal continuous training)
Localized content and software has to meet the specifications and objectives of the
context within which it will be deployed.

Technical Translation 
The localization process is largely based on technical translation as described above
through cultural adaptation.
Technical translation is in contrast to literal translation. It covers not only the translation
of technical or technological texts but all the texts that : a) belong to human knowledge
(science, arts, economy etc) b) usually contain specific terminology.
Technical translation is a kind of interpretation for different contexts and different target
groups.

Localization Methodology
Some years ago localization of software was a simple process, concerned mainly with the
translation of manuals and the user interface. Translators with average skills were able to
manage localization projects. Today things are more complex. Multimedia applications,
including e-learning applications, are more sophisticated and there has been a dramatic
increase in the number and diversity of users of multimedia applications. High quality
applications and content are produced now by local software companies in small
countries (e.g. Greece) and the market has become more cautious of localized software
and content.
Translation is a major part of the localization process but in most cases a merely
“translated” product does not appear to have been developed within the local culture.

Stages in localization methodology
Technical translation is the first step for localization. Technical translation must begin
with a detailed reading of content, this reading must be :
Dynamic. The reader (translator) must understand the meaning instead of a
simple reading.

Part of the appropriation process of the terms and relations of the

terminology included in the content.

Dynamic reading is not just an understanding of the content but can be

extended to include a search of related documentation and a complete
understanding of the subject of the content. For example you cannot localize
elearning content for total quality management without an understanding of the
theory and practice of management.

Knowing only the language that the original content is written in is not enough for
localization. The key issue is not the words and phrases but the meaning. In technical
translation the literal translation of words and phrases is not important. What is important
is the transfer of information to a specific target group that has no access to the original
content. For example the English phrase “b Cep type variable stars” could be used in
Greek using the term “Kifides”, a term that preserve the meaning of original phrase.

The following diagram illustrates this process.

56
 Meaning

Target Language
Source
Language

Terminology
In technical translation terminology is an important issue. The use of the right terms can
be a nightmare for translators. There are two key points to consider regarding
terminology:
• Often a term in the source language corresponds with more than one term in the
target language or vice versa.
• Some terms, especially in Information Technology, have been recently ‘invented’
and there is not an equivalent in the target language.

What is localized
The localization of el-earning content is applied to different kinds of contents including:
• Texts
• Video narratives or subtitles
• Descriptions of images
• User interface commands and menu items
• Set up information and instructions
• Help information
• Error messages
• Documentation
Attention must be given to:

• Terminology: use of official dictionaries is imposed

• User interface commands and menu items: translation must provide the term
corresponding in the appropriate function
• Compatibility: the same terms must be used in all parts of the content (text, help
text, documentation etc)
• Style: the style must be consistent throughout the content

57
Culture Aspects
Cultural adaptation is one of the most important issues in e-learning content localization.
Cultural adaptation is not only an issue of translation but is something more. In some
cases e-learning content must be differentiated to meet the cultural preferences of the
target group and language.
Attention must be given to:
• Idioms
• Humour
• Gestures
• Pictures
• Sounds
• Fashion
• Religion
• Values and symbols
• History
• Laws
• Colours
• Ethics and morality
The list above is only illustrative. Every type of content has its own needs for cultural
adaptation during localization.

Technical Aspects 
Beyond translation and cultural adaptation there are some technical aspects must be
considered in el-earning content localization.
• Character encoding
• Fonts
• Dates and numbers format
• Measures
• Rearrangement of shapes, images, texts
• Replacement/modification of images including text
• Rerecording of narrations, lip­synching
• Indexing and sorting of data

58
References
Folaron, D. (2002). Taking an E-Learning Project Across Borders
Dunn, P. and Marinetti, A. (2002). Cultural Adaptation : Necessity for Global eLearning.
Retrieved from LineZine (http://www.linezine.com)
Moyer, L. (2001). Is Digital Learning Effective in the Workplace?
Eveland, JD. (2001). Cultural Adaptation : Design effective eLearning across national
boundraries. Retrieved from LineZine (http://www.linezine.com)
“Effective Localization SDL International”, Polylang (http://www.polylang.com)
“Localization process overview”, CLOCKWORKS, (http://www.clockworks.com)
“How does localization differs from straight translation?”, Localization Industry
Standards Association (http://www.localization.org)

59
 Prospects for the development of software that truly
supports collaboration and learning in SMEs

Graham Attwell and Mike Malloch, KnowNet (UK)

The Learning economy

Digital technologies have been a major driving force behind the profound changes in
work organisation, production and society over the last twenty years. These changes have
led to what economists characterise as the knowledge-based economy in which the
knowledge of individuals and organisations is critical to innovation and economic and
social development. Whilst previously initial education and training were seen as
providing the basic skills and knowledge required for work and for participation within
society, the new social and work forms require continuous updating of personal and
collective skills and knowledge – lifelong learning.
Lundvall and Bórras (1999) argue that the ‘learning economy’ is a more appropriate term
than the ‘knowledge-based economy’ for economies in which specialised and codified
knowledge has a very short life-span. It is the ability to learn how to create new
knowledge, and how to adapt to changing conditions, that will increasingly determine the
performance of individuals, firms, regions and countries (see Lundvall and Bórras, 1999:
p. 31).

E-learning
Just as digital technologies have been a key force in driving economic and social demand
for new knowledge, they have also been hailed as a potential means for addressing the
problems of continuous re-skilling by developing a lifelong learning infrastructure. e-
learning, the application of Information and Communication Technologies to curriculum
and pedagogy, has been seen as providing universal access to information through
flexible and ubiquitous learning environments open to all.
Despite spawning a number of new technology companies and numerous government and
European sponsored initiatives and programmes, the uptake and efficacy of learning
using these new technologies has been less than convincing. The development of e-
learning has been dominated by the metaphors of the virtual classroom and the virtual
university, an over obsession with technologies, and a focus on distance applications of
existing learning opportunities; much less attention has been paid to the diffusion of
learning through new processes in wider social contexts. There has been very limited
attention to vocational and occupational learning, or to the development of e-learning
environments in less formal learning contexts. Research suggests that most learning - for
good or for bad - takes place in everyday life and work social situations (Nyhan et al,
forthcoming). In other words, most of our learning is informal, and takes place in a
variety of social contexts. Furthermore, anecdotal observations point to the likelihood
that e-learning may be at its most powerful in less formal and more situated learning
environments.
Work carried out in various social settings - particularly in small and medium-sized
organisations - plays a very important part in people’s lives. Learning, from an economic,
human and social point of view, has to be embedded in the fabric of all work
organisations. If e-learning is to make a contribution to changing the traditional learning

60
paradigm (institution-based; phase and stage-related), e-learning must become embedded
in the work organisation. The use of ICT (in a broad sense) for learning is considered a
major factor in implementing the paradigm of lifelong learning, and in providing staff
from SMEs in particular with access to continuing vocational training. ICT-supported
learning is expected to help in achieving the following objectives:
1. to increase access to learning opportunities through increased flexibility of
delivery modes (flexible learning) and/or by overcoming geographical barriers to
participation (physical access);
2. to enhance the quality of the learning experience in terms of content and/or
teaching
3. to increase the efficiency of the organisation by reducing costs and/or
increasing productivity
However, existing studies show that, whereas many major companies have initiated
programmes and facilities to deliver e-learning for continuous vocational training, in
particular for professional development, e-learning has until now had only a limited
application in small enterprises. Within SMEs, e-learning projects have tended to focus
on providing networked access to virtual classroom type environments or to distance
learning supported by computer-based materials. These small-scale studies have drawn
attention to a troubling lack of reliable knowledge about the current position regarding
the use of e-learning in enterprises, particularly in SMEs.

E-resources
David Guile (forthcoming) has suggested the need to move away from the present
concept of e-learning – based largely on the virtual classroom and the production of e-
learning materials - to the idea of e-resources. E-resources points to the learning potential
of the ICT technologies increasingly being used throughout the workplace, even in
SMEs. ICT may be used for communication with customers, for stock control, for
progress checking, or for ecommerce (increasingly used in business to business
transactions (B2B) and less so for business to consumer transactions (B2C)). Such an
approach would embed learning firmly within the social processes of the workplace.
Work based learning should cease to be an independent activity taking place (by location)
in the workplace, and become part of the process of work.
This approach demands a paradigm change in approaches to curriculum design and
understanding. Although many researchers have promoted a change from course driven
learning to individual learning programmes, or from individual learning to the learning
organisation, they have generally failed to address the need for new methods in
curriculum design and development. Most attempts to develop ICT-based learning have
similarly failed to develop appropriate new models of pedagogy and curriculum
development. Indeed, it might be argued that most ICT-based learning has been
regressive in promoting more rigid, course-driven learning. Why should this be so? We
suggest that the major problem lies in a failure to understand the relationship between the
media being employed - predominantly learning platforms and groupware technologies -
and the issues of pedagogy and curriculum. We have argued elsewhere (Attwell, 1999)
that all software platforms have an inherent pedagogy built into their fundamental design
constraints and decisions. Different platforms support (or fail to support) different ways
of learning. In many cases ICT is used most effectively simply for information exchange,
and when learning does take place it does so outside the special ICT e-learning
technologies provided.

61
There is a need to look more closely at how different software platforms and applications
support learning, and to develop a better understanding of the pedagogic processes, which
take place when learners engage with these technologies. In this paper, we attempt to
examine the use of collaborative environments for knowledge sharing, and to ask how
existing software environments (especially software used for communication within
enterprises) support collaboration. We examine the strengths and weaknesses of such
environments for collaboration and learning. We then go on to discuss what kind of
systems might be developed to strengthen knowledge sharing and learning within
workplace environments. We are not suggesting that the development and
implementation of new software can by itself transform the workplace into a rich learning
environment. We are suggesting that there is a need for software developers to work
together with SMEs and SME associations to iteratively develop and test new software
tools and systems that can facilitate collaborative knowledge sharing and development in
real-world work contexts.
We contend that the focus on e-learning software and platforms (and the hype around e-
learning) has diverted attention away from the learning potential of email and groupware
products that are being used to a greater or lesser extent in larger corporations and in
some SMEs. We secondly argue that because of the ‘distance’ between developers and
users there has been little attention paid to the evaluation of software - not in a technical
sense, but in terms of its potential for enabling knowledge development and learning.

What is wrong with existing collaboration software?

Before delineating the limitations of existing software, let us praise what is good about it.

What is right about existing collaboration tools

It is evident that existing tools for online working and knowledge sharing have made a
wide impact, if not a deep one. We divide our discussion of these successes into two
categories:

Ubiquitous, standards-based tools

Almost everyone in the industrialised world now has some access to tools and content-
delivery environments based on industry standards: email and web pages are familiar to
everyone, and have changed many people’s working practices.
The great merit of these tools and environments is that they are based on thorough suites
of interoperability standards, and so their scope reaches far beyond individuals’
organisations to allow the spontaneous emergence of virtual communities of interest and,
to some extent, practice.
To a lesser extent, yet to be standardised but widely available, peer-to-peer instant
messaging, chat, video conferencing, internet telephony and shared whiteboard
applications are used by many people to enhance synchronous communication at a
distance.

62
Some more recent standards, like XML, RDF, SOAP,
RSS, and emerging lightweight standards for
web-logging communities, are beginning to
enable solutions like shared newsfeeds,
interchangeable representations for online
content, and distributed commentary, linking
and description of content from many sources.

Groupware and knowledge management tools

For more than a decade, special commercial environments have existed to aid people in
organisations to communicate and to organise knowledge resources.
Groupware is a term used to describe products, like Centrinity’s FirstClass and Lotus
Notes, which provide a suite of collaboration tools such as shared conferences (shared
email folders), calendars, resource spaces and chat rooms.
Non-commercial and open-source products also exist for shared bulletin boards and
topic-centred discussion arenas, but these are less functional, with less powerful
administration, customisation and moderation tools than the commercial products. A
standards development or open-source movement to create free, highly functional,
standard environments and tools for groupware would be a very useful step forward.
Knowledge Management is a more recent term applied to environments in which
companies can organise corporate document repositories and present a front end
combining access to these with groupware functionalities, often also allowing users to
customise their view of these resources.
Content Management is a related term applied to tools for expressing, archiving,
versioning, managing and delivering large repositories of documents.
These environments provide a significant step forward from ‘standard’ email and web
pages, mostly by organising messages and content so that users can easily access shared
versions when they need them. Standard email does not allow shared copies of messages,
and forces each user to undertake the task of organisation independently. Standard web
pages make very poor document repositories, since they are designed for appearance
rather than content, and are hard to edit or version-manage ( though we note that these
failings can to some extent be obviated by increased adoption of ‘modern’ web-standards
based techniques for ‘semantic’ markup in html or xhtml, well-separated from
appearance-control using Cascading Stylesheets)
We ourselves are avid users of such systems and would not deny their usefulness for
certain purposes if carefully maintained and moderated.

63
In addition to groupware and knowledge management environments,
there has recently been a proliferation of ‘virtual learning
environments’ (there are many similar terms, such as ‘managed
learning environment’ and ‘e-university’). These are, essentially,
a combination of some groupware and some knowledge
management functionality, generally not thoroughly implemented,
along with some basic tools for generating course notes. Our
experience indicates that a great deal of human intervention is
required when employing these tools in educational settings, and
that very little value is added to the learning process regardless of
the effort expended; what is useful in disciplined work
environments seems to be much less so in learning.
Why is online collaboration and learning still so sparse?
In spite of the universal availability of email and web pages, and even with the careful
deployment of groupware, knowledge management and ‘virtual learning’ environments,
communities of users do not enjoy the rich interactive experience we feel is essential
before the internet can become a medium for knowledge work and learning.
We briefly list below some of the problems as we see them.

The ‘place’ metaphor is weak

Whether exhibited in ‘site’, ‘location’ or ‘discussion space’, the overwhelmingly
dominant metaphor for engaging with web content and messaging is place: ‘Where’ do
you want to go today? ‘Where’ should I send this report? ‘Where’ should I look for this
piece of information? ‘Where’ did I put that message about last month’s meeting?
Note that, as natural as this sounds when thinking about the physical activities associated
with knowledge work and learning, it is in fact profoundly unlike the terms that naturally
describe the activities of knowledge work and learning themselves.
What, why, who, how… these are more natural terms. In fact, when analysed closely, we
lack explicit terms for most of the cognitive and inter-subjective motifs of topic-setting
and shifting, contextualising, assertion, reflection, consensual edifice-building, etc. This
should not be surprising – people are individually and collectively too embedded in these
processes to be aware of them. But we must not allow the spurious salience of the place
metaphor to stop us from reflecting on the absurdity of applying it to the design of
environments for manipulating knowledge in a non-located information-based medium
Part of the problem with the place metaphor is that it forces us away from seeking ‘what’
we want to know, or expecting appropriate background resources to be made salient to us
by the system when we express ‘what’ we are engaged in with respect to ‘what’ issue.
We may know that the system is too dumb to do that in existing technologies, and that
our best bet is to look a ‘place’ where we can start tunnelling through hyperlinks to get at
a fact or opinion. But this need not always be the case.
Another problem with the place metaphor is that knowledge – even as expressed in web
pages or documents – is always distributed, and thus never to be found in one ‘place’. If
knowledge could be expressed so that a dumb computer could assemble what we want to
look at from distributed ingredients, instead of people having to repeat this task with long
sessions of mouse-clicking, life would be much easier. If authors could express
themselves in the context of such a system, so that background and related materials were

64
automatically available to their readers, a lot of repetition and circumlocution could be
avoided.
It is not just with respect to documents and crafted content that the place metaphor is
inappropriate. Electronic messaging, though it currently seems naturally about ‘sending’
to a ‘place’ (or an ‘account’), could be much more powerful if notions like ‘about’ and
‘assertion’ could organically create the system’s links between ‘messages’ (brief ad-hoc
contributions to a conversation), content and activities, as well as notions like ‘to whom’
and ‘in reply to which item in someone’s in-box’.
Please note that we are not implying any reliance on the representational techniques and
complicated inference engines of Artificial Intelligence. Sadly, we also have bitter direct
experience of the futility of that approach. The alternatives we propose are simply to
allow interfaces for people to structure and contextualise their writing, resource-collation
and messaging in meaningful but sharable categorical terms that can be processed by
computers, to allow messaging and content to be discoverable and collatable
automatically from their semantic properties, and to allow content and discourse to have
common interfaces in each others’ context.

Expression of meaning is still textual

Web pages and email messages are both based on simple standards for the transmission
and display of text characters. This is why universal standards have succeeded for them;
no-one had to agree about how to express meanings, just characters. The html standard
has a very few, very broad categories like heading level and list-item, and the email
standard allows a subject separate from body, but there are no conventional ways to
denote more specific semantic types or gestures.
It would be very hard to reach agreement on standard ways to ‘tag up’ meanings, and we
do not propose that standards should exist for this, but we note that there are some useful
ingredients in place for at least standardising a non-textual medium for expressing
meanings:
- the XML standard allows the exchange of arbitrary tree structures, and of the
grammars on which they are based, in a widely accessible way
- the Dublin Core metadata standard allows the expression of basic descriptive
assertions
- various domain-specific sub-standards are in development (usually based on
XML) for expressing meaningful kinds of gesture, for instance chemical formulae
and mathematical equations
- technology exists that could allow a standardised platform for expressing,
sharing, and discussing a variety of community-specific semantic structurings
while using common tools for searching, organising and displaying structured
content and metadata
What might it be like if there were conventions for expressing meanings directly, rather
than as strings of text characters? Computers are very bad at processing meanings, but
very good at storing and processing abstract structures. We would not expect computers
to do anything with meanings (outside of restricted areas like simplifying formulae and
querying databases), but they can display and organise structures and categorical types to
human users on the basis of rules, and they can allow humans to make searches, organise
content and specify relationships in terms of abstract types and structures. Fine-grained
expression of meaning would be cumbersome in such terms, but broader-grained
expression could be more convenient than prose to generate (think of the difficulty

65
authors face in turning the ‘mind-map’of ideas and relationships they are trying to convey
into linear prose or bullet-points).
We list below some of the consequences of using text characters (organised into files or
messages) as the standard for conveying meaning.

Hard to find appropriate content

The text-as-standard becomes an obvious problem for most people when they are trying
to search the web. Standard web searches must be based on strings of characters found in
the target documents; as we all know, thinking of useful search strings is a dark art and
not always a useful one.
‘Metadata’ is a term used for categorical or semantic assertions made about pieces of
content. Many web users are becoming familiar with the term, as frustration with
character-string searches grows.
Of course, services have emerged which attempt to apply some categorical structure to
web content:
- many pages have been sorted into broad categories by search engine staff, or
by volunteers of the open directory project
- link sites, which painstakingly attempt to provide a common entry point for
content related to a particular issue
- many authors and organisations now embed Dublin Core metadata in their web
pages, sometimes alongside other metadata terms
- community or organisation metadata repositories, like link sites, can apply
semantic terms to web resources)
These services suffer from being based on structures that are either shallow (large-scale
categorising efforts, Dublin Core) or sparse (community repositories). It would be much
easier to leverage the human effort of categorisation and semantic tagging if documents
were expressed with some semantic structure in the first place. For many documents or
messages, structured authoring could provide enough ‘self-metadata’ to allow powerful
searching.
Note that the same problem applies to email or groupware messages. In heavy traffic,
important message content quickly becomes lost, and text-string search is often incapable
of retrieving it (think of the head-scratching you’ve done to try to remember a phrase or
exact term from a message so that you can search for it in a folder or conference).

Content does not interoperate meaningfully

Web content is widely used for educational purposes, but a constant problem in the
design of collaborative learning environments is that web pages are monolithic and
stupid. Monolithic in that they can only be delivered all at once, replacing whatever
content had previously been displayed. Stupid in that they cannot send information to
their environment (whether about user actions, or even just about their own properties). It
is possible to impose special constraints on web-page designers to make it somewhat
easier to assemble them into learning sequences and environments, but this method
defeats the point of standards, and is in any case fraught with technical difficulties.
A second problem is familiar to those who have created educational web content.
Whereas the dream of hypertext was to allow a universe of content to interact, putting an
omniscient tutor over every student’s shoulder as she acted on your content, the reality is

66
that linked or background content has to be hand-crafted (or at least hand-linked) by the
author.
How much better it would be if authors had only to create that part of their content which
is unique to their purpose, with content from other sources accessible on the basis of its
semantic properties, and able to be woven into displays and sequences on the basis of its
structure.
How much more interactive the user experience could be if small chunks of content from
whatever source could be woven into an integrated and managed client environment so
that their actions within the content could be processed by the user’s choice of software in
the context of the user’s preferences, peers and other activities being undertaken in the
session.

Hard to link related content and conversation

A common problem in the design and use of groupware systems for learners is that it is
very hard to sustain electronic discussions around resources.
In face-to-face communication, even by telephone, humans have a wealth of techniques
for indicating topic and for ‘pointing’ at objects, resources, themes or goals. In electronic
messaging, these techniques are not available, and other devices must be found,
exploiting the graphical user interface. Embedded URLs are very useful for pointing to a
resource in the first instance, but very bad at tracking the resources or versions involved
in a sustained discussion. URLs are given the same salience in groupware systems as any
other text within a message body; relationships and links are not foregrounded in the user
interface.
If it is difficult to point to a web resource from a discussion, it is even more awkward for
users to point to
- one discussion thread from another
- one piece of content from another
- a discussion thread from a piece of content
Two-way links are, at present, almost impossible to create.

‘re;’is a weak basis for discourse structuring

The ‘reply button’ is, of course, the generator of most electronic message traffic, as the
frequency of the prefix ‘re:’ in email subject headings attests.
Unfortunately, the progress of discussions is often lost in this succession of ‘re:’s’. The
reason is that, while there is only one type of ‘reply button’, users will in fact be
expressing many different kinds of response, some of which move the discussion forward
or away from the original thread; this variety and movement is lost in a homogeneous
chain of ‘re: re: re…’.
This makes it hard to follow threads in heavy-traffic sites, and very hard to return to old
threads or enter them as a newcomer.
Note that it is technologically possible to design groupware that provides special ways of
replying with conversational gestures.
A second source of information loss in the reply chain is that, while messages typically
contain many and diverse ideas and points, it is only possible to reply to the entire
message, not its sub-content,
This makes it impossible to follow the strands of threads as they branch.
Structured messages and structured linking of new message content to existing content
and discussion would allow individual points to be organised transparently within
message content and responded to individually. Instead of homogeneous linear threads, it

67
would be possible to create diverse branching trees and webs of message content and
links.

Content creation, deployment and management is

technical
Because HTML is designed to describe the appearance, as opposed to the meanings, of
content, authoring web pages is to some extent a technical process, even when graphical
tools are used. Deployment (posting to the web) and management (editing, controlling
navigation, version-control) are even more technical; usually authors and editors have no
direct control over their content at all once it enters the deployment workflow.
This creates many problems for authors. It also separates those with knowledge of
content from control over the navigation and link structure of larger content blocks, and
makes flexible organisation of structure and content very difficult.
Readers, too, lose control over content needlessly. If content were authored, stored,
organised and delivered in terms of its meaningful properties, users could flexibly control
the way they want to view it. Currently, users have very little control even over the
appearance of a web page they are viewing.

Structure is sparse, shallow and mostly accidental

Content and discourse on the internet is, of course, structured already. Roughly, the
categories are site, page, groupware conference, discussion thread, etc.
But this structure is
- inflexible
- largely accident of history of content and message creation, and thus
uninformative
- shallow both in tree-depth and in semantics – there is little meaningful range of
structuring decision that can be applied to standard web content and messages
-

The ‘action’ is missing from ‘interactive’

Learners and knowledge workers can look at content, but what can they do with it?
Because content is monolithically unstructured, and its link-structures are painstakingly
hand-assigned during authorship, there is nothing in the nature of the content itself that
invites action from a user, or that can be actively related to, or aggregated with, other
content. ‘Interactivity’ has come to mean either action on remote databases through data
entry forms, or action on highly expensive islands of animation or branching movies. If
content comprises vertical books, users prefer to read it on paper.
It is the constant frustration of workers trying to elicit usership of e-learning platforms
that their users find so little to do with the content they are presented with that they
engage very shallowly and infrequently.

Discussion divorced from content

Conventional systems make a sharp distinction between conversation and resource (the
popularity of listserver archives as a public technical resource reveals a low-tech example
of the internet community spontaneously disagreeing with this distinction). We believe

68
that a large part of a community's 'know how' is knowing what people talk about when
they talk about a field or a work practice.
If both content and messaging were authored and accessed in structured ways, the
distinction would naturally blur. Content could grow in small chunks with rich relations
to networks of discussion and content.

Key issues and ideas

Sharing & structuring: Metadata, XML & semantic

expressiveness
We feel that the key requirement for structured authoring, metadata and messaging to
succeed is an architecture for sharing and reconciling diverse structural ‘grammars’ and
semantics.
Our experience with metadata for online learning resources makes it very clear that there
is no ‘correct’ metadata schema. Users will only feel comfortable and empowered when
they can directly influence, and immediately override, the structures they use to express
or describe their meanings.
This of course creates a problem for interoperability. But that is only a technical,
architectural problem – we can design systems to help maximise both
coverage/interoperability and richness/expressiveness. Such systems will not be
automatic. They will have to provide good interfaces for users to discover, discuss, refine,
reconcile and decide on schemas, and for information professionals to design mappings
and canonical schemas for common purposes. Tools must be able to seamlessly deal with
a wide range of schemas (for instance our own advanced metadata tagging and search
tools configure themselves on the basis of XML schemas chosen by users).
Users must have a wide range of structured editing tools present to hand in their
appropriate contexts, together with a rich ability to find and link in related content.

Structured communication
Electronic communication is changing the way people work, enabling remote
collaboration and accessible, self-documenting discussions. As uptake and usage of
email and electronic groupware increases, though, the limitations of current protocols and
tools are creating real problems for sustained collaboration.
One problem concerns the unstructured nature of email. While text messages can carry
important meanings when their context is shared between writer and reader, they are
harder to understand outside of that immediate context. Nor is it straightforward to
search for those meanings as time passes after their initial receipt, or to combine
meanings from multiple messages into new ideas or assertions, or to link those meanings
to other electronic environments. At present, everything depends on text, and on
hierarchical organisation imposed on message storage (whether in email clients or
groupware environments).
This problem only exhibits itself at larger scales of usage and higher expectations of
sharing and building on an organisation or community's discussions. For simple,
lightweight use, simple email is effective. At slightly larger scales and slightly higher
functionality expectations, conventional groupware makes a big difference, but, even
when very carefully moderated and maintained, groupware systems become less useful
and accessible as they grow in success.

69
Structured messages could be very useful in overcoming the frustrations users experience
with large numbers of text messages. By 'structured', we mean that different aspects of
the communicative act are represented distinctly in the messaging and message-browsing
system. In particular, structured messaging could replace the concept of 'thread' (which
organises messages by a sequence of 'reply' acts) with richer, multidimensional concepts
like 'raising issues', 'solving problems', 'tracking progress', and 'linking related material'.
Structure in the messages could be represented semi-graphically to message readers and
composers; more importantly, the relationships among messages - and between messages
and other resources and documents - could be graphically represented.
By providing an environment in which many small components and content-nodes were
already available for re-use, embedding and cross-linking, and in which simple 'gestures'
could replace textual elaboration, message composers could eliminate much of the
longwinded re-expression and periphrasis that makes context-setting so difficult in
existing email.
Readers of structured messages could avoid having to scan long passages of redundant or
familiar text; they could get the 'gist' from graphically depicted structure and
relationships, and 'drill in' to the aspects of content and context that they needed.
Perhaps most usefully, message structure would automatically create a form of structured
metadata allowing rich querying and alternative virtual organisation of communities'
valuable discussions.

Freeing users to create webs of knowledge, practice

and discourse
The key problem in any attempt to experiment with new internet architectures is ‘where
will all the content come from’? The web exploded with content because of its very
simple standards – anyone could create a web page (though it was very much like early
desktop publishing in requiring naïve authors to attend to fine-grained page design). If
we are to expect interestingly structured, interoperable content, authors are required to
change the way they write; we have found this to be a very difficult learning curve for
most professionals.
Our key idea is to design the environment to reduce the average ‘chunk-size’ authors
must create, and maximise the value of small structured bits of knowledge in the right
context.
This is why we feel it is key to
- provide a structured messaging environment, so that message content can
become core content
- provide a very thorough and responsive interface for finding materials and
relating them richly to new content, whether message content or ad-hoc ‘nuggets’
of knowledge.
- provide a transparent system for sharing structures, so that users can explore
rich semantic structurings for their own uses without locking themselves into non-
interoperable modes of searching and organising.

Building new knowledge on practice

The aim of these systems would be to allow employees in SMEs to develop and build
new knowledge and learning through everyday work practices and through
communication within communities of practice. Communication and learning could
become synonymous. This is not to suggest that all communication leads to learning, but

70
we do believe that learning and knowledge can be derived from the communication
processes which take place in the work environment. A further aim would be to support
the social processes of learning and overcome (at least in part the problems of atomised
and individualistic approaches to knowledge development which bedevil many e-learning
platforms.
Three key steps would be required to move forward towards the approach we advocate.
Steps for the future
1) Research: More research is needed into the nature of learning and knowledge
development in SMEs. This should include an examination of the links between
work organisation, technology and learning, and of how technologies and work
organisation can be optimally designed to support learning processes.
2) Standards: New standards are needed for interoperability of software and for
curriculum design. The present nascent standards in the education and training
sectors have focused on the specification and description of learning materials,
rather than on pedagogies and learning processes. Such standards derive from the
course-driven paradigm, focusing on number of hours, levels, outcomes etc. By
their nature, the standards exclude the use of many software applications in
enterprises. Concepts of functionality and interoperability spaces based on
pedagogy – on the learning process – could support far wider notions of learning
and curriculum development. We note that progress towards such aims will not
be swift or easy since it will have to involve both innovation and co-operation on
a large scale.
3) Co-development: We need new forms of partnership between software
developers and educational researchers and practitioners. Much of the global
investment in e-learning over the past five years has been expended in the
production and implementation of stable but limited e-learning content-delivery
platforms, with much less investment in exploring new ways of harnessing ICT
within the learning process. Too much e-learning software is written because it
‘can be’, while very little production-quality effort has been applied to creating
the software that ‘must be’. Software developers need to be involved in the
practice of learning. At present there is a wide skills gap - widely different
occupational profiles - between those who develop learning platforms and those
who support learning practice.
In Europe, at least, there is considerable public funding being spent on e-learning. A
refocus towards workplace learning (and away from the virtual classroom), and increased
emphasis on research and practice in learning processes at the workplace, together with a
concerted attempt to build deeper and earlier working relationships between developers
and practitioners, could support a first step towards the paradigm change we call for.

71
A framework for the evaluating of e-learning
Jenny Hughes, CRED (UK) and Graham Attwell, KnowNet (UK)

Introduction
The development of e-learning products and the provision of e-learning opportunities is
one of the most rapidly expanding areas of education and training. Whether this is
through an intranet, the Internet, multimedia, interactive TV or computer-based training,
the growth of e-learning is accelerating. However, what is known about these innovative
approaches to training has been limited by the shortage of scientifically credible
evaluation. Is e-learning effective? In what contexts? For what groups of learners?
How do different learners respond? Are there marked differences between different ICT
platforms? Does the socio-cultural environment make a difference? Considering the costs
of implementing ICT based training, is there a positive return on investment? What are
the perceptions of VET professionals? What problems has it created for them?
E-learning is also one of the areas that attracts the most research and development
funding. If this investment is to be maximised, it is imperative that we generate robust
models for the systematic evaluation of e-learning and produce tools which are flexible in
use but consistent in results.
“Although recent attention has increased e-learning evaluation, the
current research base for evaluating e-learning is inadequate…Due to the
initial cost of implementing e-learning programs, it is important to
conduct evaluation studies.” (American Society for Training and
Development 2001).
The Capitalisation report on the Leonardo da Vinci 1programme, one of the biggest
sponsors of innovative e-learning projects in European VET, also identified the lack of
systematic evaluation as being the major weakness in e-learning projects.
However, whilst some have been desperately seeking answers to the question `What
works and what doesn’t work?’ and looking for ways of improving the quality of e-
learning, the response by a large sector of the community of e-learning practitioners and
by the technocrats in particular, has been a growing preoccupation with interoperability
and regulation of platforms and models. This almost certainly stems from the
bureaucratic confusion between standardisation and standards and a belief that tightening
the first will somehow improve the second.
This paper is written from the perspective of professional evaluators and evaluation
researchers, rather than VET researchers or even e-learning practitioners, although we
have been at various times involved - and are still involved – with both of these fields. It
is underpinned by a set of assumptions about the evaluation process and also based on
personal research evidence that suggests that standards are more likely to be improved by
diversity, flexibility and experimentation than through standardisation.
The first section outlines the principles and assumptions on which our evaluation work is
based. The second section is a review of over 200 evaluation reports on e-learning with
the focus on identifying the type of report rather than its content. This is followed by a
proposed framework for reporting on and classifying e-learning evaluation together with
some lessons we have learned from our own evaluations.

72
Basic Principles
We recognise that the purpose of evaluation may be primarily developmental, concerned
mainly with improvement and learning, or may be concerned with accountability and
justification and more inspectorial in focus. However, we believe that in both cases
• Evaluation is an essential element in the design and planning of any e-learning
programme or innovative process
• Evaluation is integral to e-learning activities and not `bolted-on’
• Evaluation should span the whole lifecycle of the programme and should be
formative as well as summative.
• Evaluation should be client centred, based on a non-dependency relationship
and leading to long term client autonomy and sustainability
• Evaluation should recognises the diversity of stakeholders and respond to their
different needs by offering a wide range of evaluation products, tools and
processes.
• Evaluation is a skilled intervention and a specialist field of knowledge and
practice
• Evaluation should be ethical, professional and responsible
• Evaluation should be informed by a range of different approaches and
theoretical perspectives to ensure congruence between the evaluation process and
the policies, processes and practices being evaluated. Thus, the evaluation of e-
learning should make use of the opportunities created by the technologies.

A taxonomy of e-learning evaluation

A web-based literature survey of American, European and Australian e-learning
evaluation reports in English, French and German revealed that the existing research is
very limited and can be grouped under seven main headings.

• Case studies of specific e- training programmes

For the most part these are descriptive rather than analytic or predictive, predominantly
American, mainly located in a Higher Education rather than vocational training
environment and focussed on the ‘virtual classroom’ model. They also tend to be
restricted to particular subject areas, in particular I.T, languages and engineering
disciplines. (This is not necessarily to say that e-learning is restricted to these areas,
rather that they are over-represented in evaluation reports.)

• Comparisons with traditional learning

There are some (but surprisingly few) systematic studies that compare e-learning
effectiveness with traditional learning and which are empirically robust. Those that exist
are mainly small-scale studies, often using a matched pairs design and are frequently of
very specific instances of e-learning in which the e-learning methodologies are
idiosyncratic and the conclusions are non-generalisable

• Tools and instruments for evaluation of e-learning

There is an abundance of literature detailing tools for the evaluation of e-learning.
However, these are mainly divided into two types. Firstly there are many on-line data

73
gathering instruments for assessing, typically, the user interface characteristics of
software (e.g. student perception questionnaires) or secondly, there are devices to record
and analyse useage by duration and frequency of log-in, pages accessed, user profile etc.
Many of these are sophisticated in their design and ingenuity but lack guidance on
interpretation and analysis.

• Return on Investment (ROI) reports

There are surprisingly few ROI reports, considering the huge investments into e-learning
at all levels. The majority of those that exist draw mainly from industry based examples
and are written from an HRD perspective. The conclusion is inevitably that the
investment was cost-effective and represented value-for-money but often the savings are
defined in efficiency rather than effectiveness with no long-term impact analysis that
takes account of unintended outcomes and consequences. It is also difficult to compare
figures across reports because the distinctions between net and gross costs, capital and
revenue costs, displacement of existing funds, costs over time etc. are often blurred or
missing. Many ROI type evaluation reports appear to be justifying investment rather than
evaluating it and more geared to an audience of shareholders rather than researchers.

• Benchmarking models
There have been several attempts to generate sets of criteria for quality assuring e-
learning. However, these tend to be skewed towards proposing quality standards for e-
learning systems and software which often disregard key variables in the wider learning
environment or are based on criteria associated with evaluating traditional learning
processes (and which disregard the technology) or criteria associated with measuring
learner achievement through traditional pedagogies. An additional problem is that the
designers of these benchmarking systems are often locked in to a particular model of e-
learning which limits their transferability.

• Product evaluation
By far the greatest number of `hits’ on evaluation of e-learning are reports describing
(and extolling the virtues of) particular education software. The vast majority of these
reports are commissioned or published by the software developers. This is not to question
the usefulness of these reports or necessarily to doubt their validity but evaluation of `de-
contextualised’ software is not an acceptable substitute for the rigorous evaluation of e-
learning systems.

• Performance evaluation
Scrivens (2000) in the USA, uses the term `performance evaluation’ for what would, in
European terms, be called student assessment. Whilst it is true that an examination of
student performance is a powerful indicator of the effectiveness of e-learning, it is by no
means the only one. Moreover, a survey of reports on performance evaluation in the
context of e-learning were mainly concerned with on-line tools and instruments for
examining knowledge-based learner performance and could therefore be categorised
under that heading.

Analysis of results of literature search – the gaps in

the `research market’.
Although the existing literature is limited, there are, nevertheless, some significant
findings and key issues to address. Not surprisingly, the validity and usefulness of the
research into the evaluation of e-learning is often limited by the agencies that drive it.

74
Firstly, most of the current evaluation at the business impact level has been driven by
clients or buyers of e-learning (those who are funding the project) rather than by the
designers, developers and front line deliverers and there is evidence in many cases that
they are seeking to justify their investment. The evidence also suggests that this is even
more the case in the public sector than in the corporate sector and more the case in large
companies than in SME. (Changing Technologogical Management : The Evaluation of
the Further Education National Learning Network, 2000 DfEE UK)
Examining the explicit and implicit political and economic agenda of key agencies in the
evaluation of e-learning is a major area of investigation. Is the purpose of evaluation
about justification and valorisation or about improvement ? Do we really want answers
and if so, what ‘currency’ would the answers have ? How would funding agencies react if
evaluation reports were published claiming that e-learning was not effective, that e-
learning projects had actually failed ?
Secondly, available evidence, thus, far suggests that traditional classroom instruction
yields a more favourable learner response than e-learning solutions. (Kirkpatrick 1959 :
Level 1 Evaluation). This issue represents a perplexing problem for proponents of e-
learning. (e.g.Scrivens M 1999) It also raises the question of whether evaluation of e-
learning compared with traditional learning should be the real issue or is it evaluation of
e-learning within itself ? (And similarly, between different e-learning platforms). Further
research is needed to explore these two fundamentally different perspectives and generate
reference materials which will look at the strengths and limitations of norm-referenced,
criteria-referenced and ipsitive-referenced models of e-learning evaluation.
Thirdly, e-learning has been shown to be as effective as traditional face-to-face learning
over a parallel series of comparative studies with American university students over a
range of science based subject disciplines (U.Ohio Centre for Evaluation Studies 1998
-2000). While recipients of face-to-face learning have expressed more satisfaction
(`Level 1’ evaluation - Kirkpatrick 1959) with traditional learning solutions, the learning
outcomes (Level 2 evaluation) are not different for participants of e-learning
programmes. Another key area for investigation should be whether these results are
replicated in a European context, for non-university students and in vocational rather than
in academic subjects.
Fourthly, many researchers have claimed that the same evaluation strategies and
processes utilised in other types of evaluations can be applied to e-learning programmes.
This may or may not be the case. However, a re-examination of widely used models and
benchmarks is warranted (e.g. the ‘1-5 Levels of Evaluation’ proposed by Kirkpatrick)
and any inconsistencies and limitations in an e-learning evaluation environment identified
particularly for their applicability and potential for adaption or refinement for the
evaluation of e-learning in European VET.
Fifthly, the ROI studies indicate a positive return for companies implementing e-learning
programmes. (Costs of Networked Learning, 2000-1 funded by JISC). Although most
studies show a positive return based on cost reduction alone, (although often expressed in
terms of revenue rather than capital expenditure) ROI studies need also to include
analysis of benefits and ‘hidden’ costs for example, training the trainer staff costs, time
away from the workplace costs. New research is needed which attempts a broader
analysis and, in particular, will apply the ROI model to VET providers in the public
sector.
Sixthly, there is evidence of a growing practice of building evaluation into an e-learning
process through the use of on-line tools that assess students’ perception and performance
based on the belief that this can save time as well as money. This notion should be

75
examined from the perspective of the pedagogical assumptions underpinning it and the
robustness and usefulness of the data generated in this way.
Finally, most of the credible, holistic evaluation of e-learning has been based
predominantly on an evaluative approach based on systems theory or using a positivist-
rationalist approach. This is actually the case in most evaluations of VET programmes
but the limitations of `systems theory evaluation’ (feedback and error detection) may be
more significant in e-learning than in traditional learning. The relevance of this approach,
particularly at policy level should be challenged and alternative theoretical bases
explored using some of the models generated by Van der Knaap (1998 –2001) and by
Elliott Stern at the Tavistock Institute on policy evaluation.
However, there is some good news. Useful empirical work is being undertaken by
Sheffield Hallam Telematics in Education Research Group (EU CI funded MOE and
REMiT projects) although this group does not focus specifically on evaluation research
nor on VET but has produced interesting models. The Oriente network for the evaluation
of ICTs in education has also generated outcomes and ideas that could be adapted to VET
rather than general education. Similarly, the Leonardo EVAL 2 Leonardo 2 pilot project
is building a virtual support environment for evaluators and an on-line database of
evaluation resources and the Capitalisation and Evaluation Research Network (Leonardo
2 network project.) brings together evaluation and VET professionals to work on
common themes. All of these projects have identified the evaluation of e-learning as a
major issue but have insufficient resources to allocate to in-depth work on this specialist
area.
What is lacking is a theoretical basis and a coherent research framework. There is little
systematic research into broad based issues and concepts, or the generation of
transferable models and processes of evaluating e-learning or into the design of tools for
analysing, rather than collecting, data. Furthermore, there are few papers written which
collate the results of the existing research and classify it in an accessible way. Nor is
there substantial evidence of work that extrapolates and tests generalisable principles
arising from the case studies and surveys or which comments on the implications or
application of these in a European VET arena.
Moreover, the gaps in knowledge impact on every level of the VET infrastructure. Policy
makers and policy influencers need greater awareness of the implications of particuar e-
learning strategies and models to make informed decision on e-learning policy and
funding. There need to be improved links between research and evaluation so that
evaluation outcomes inform the research agenda and researchers can improve the validity
of field observation. The skill base of E-learning evaluators need to be increased and
they need tools and instruments which will increase their ability to make more analytic
and interpretive evaluations of e-learning, using a greater range of methodologies. E-
learning managers, staff and other VET professionals need better evaluation products so
that the design and delivery of e-learning programmes is improved.

A New Framework
From a baseline of practice of attempting to evaluate many e-learning programmes, one
of the biggest problems has proved to be handling the number of variables which
potentially impact on the effectiveness of the programme and deciding what constitutes
dependent, independent and irrelevant variables in a given situation.
Over several e-learning evaluation projects, five major clusters of variables have
emerged; individual learner variables, environmental variables, technology variables
contextual variables and pedagogic variables. Each of these can be disaggregated into

76
more precise groups and further disaggregated until individual variables can be identified
and isolated.

Individual learner variables include

physical characteristics (e.g age, sex, physical abilities)
learning history, (negative / positive experience, level of attainment, duration, recency
etc)
learner attitude (positive / negative)
learner motivation (high / low)
familiarity with the technology

Learning environment variables include

the immediate (physical) learning environment
the organisational or institutional environment
the subject environment the

Contextual variables include

socio-economic factors (e.g. class, gender,)
the political context (e.g. who is funding /paying for the e-learning and for what reason ?)
cultural background (e.g. how highly is learning / e-learning valued ?)
geographic location (e.g. country, language, urban/rural)

Technology variables include

hardware
software,
connectivity,
the media
mode of delivery,

Pedagogic variables include

Level and nature of learner support systems
accessibility issues.
Methodologies
Flexibility
Learner autonomy
Selection and recruitment
Assessment and examination
Accreditation and certification

We are currently undertaking a survey of practitioners to test out the validity of the above
taxonomy, which is also being informed by a search of existing literature and thus is
constantly changing. The intention is threefold.
Firstly we are seeking to build a robust classification system with clearly identified levels
of aggregation, (which themselves may be context determined.) for mapping and coding
existing work into the effectiveness, efficiency and economy of e-learning irrespective of
whether this is an evaluation or an independent research study. Methodologies are cross-
referenced against the variables being studied and major areas of omission can be
identified that in turn will suggest a future research agenda.
Secondly we are using the clusters of variables for proposing and testing hypotheses. For
example, at the micro level, part of the iLab Project is testing the hypothesis that the

77
effectiveness of different e-learning pedagogies will depend on particular individual
learning histories. Another survey is exploring whether the effectiveness of particular
technologies depends on gender. At a macro level we are also interested in whether the
presence (or absence) of some individual variables or clusters of variables are more
significant than others in determining the effectiveness of e-learning and, if so, can they
be weighted in some way? Is the profile of the learner more significant than the nature of
the learning environment? Is the effectiveness of the technological solution outweighed
or enhanced by particular environmental variables? Which is more important – getting
the software right or the learner support right? Can we use multi-variance statistical
techniques (such as factor analysis) to see which variables ‘cluster’ together and the
extent to which they impact on each other?
Thirdly, we have found it a useful framework for evaluating and researching the
effectiveness of specific e-learning projects and programmes. The evaluation of e-
learning, and research into the evaluation of e-learning, has been dominated by
descriptive ethnographic studies, rather than interpretation and analyses and there is a
predominance of ethnomethodological approaches, in particular, heavily contextualised
case studies. The relatively small number of empirical studies have focussed on a limited
number of variables. The best of these have controlled for variables other than those
under study; the worst have simply discounted them. As the databank of research results
is built up, particularly as the different variables are `weighted’, it becomes easier to
identify the irrelevant variables and allow for the impact of others. It also allows
predictions to be made which can short circuit the search for an appropriate evaluation
methodology.

Conclusions
Our overarching conclusion is that the evaluation of e-learning is fundamentally the same
as the evaluation of any other learning but with particular groups of variables playing a
more prominent role and the impact of others differing significantly from their impact in
traditional learning.
Specifically, we have found that the political factors are crucial because the nature of e-
learning challenges conventional theory and established benchmarks. For example, the
completion rates are notoriously low in e-learning. E-learning is by definition easy to sign
up to and, by the same token, easy to drop out of. This threatens institutional models. E-
learning promotes learner autonomy and learner choice about how, when and where to
study. Is it surprising, therefore, that learners make decisions about when they feel they
have learned enough for the moment? Is this a bad thing? In the majority of cases we
have looked at, `dropping out’ has not been to do with the quality of the course as
perceived by the student, but a conscious decision that they wanted to stop. Quality
standards for e-learning programmes often rate student choice highly and yet still persist
in using completion rates as an indicator of effectiveness thus contradicting their own
implicit value judgements.
The social control model in e-learning is also diminished; there is usually no peer
pressure, unlike traditional learning environments and the teacher-student relationship is
weaker, a situation which is often at odds with the institutional culture.
Interestingly, most of the e-learning programmes we have looked at, particularly the most
recent, use less variety of media than much conventional teaching. For example, there is
often a sense of `failure’ on the part of programme organisers if material cannot be
provided on-line, even when recommending a book or sending an audiotape through the
post would be more effective. Much e-learning has lost the concept of `multi-media’ in

78
the search for technological sophistication. Similarly, access to a diversity of source
materials type is harder to effect on-line although quantity of material is increased.
There are many other examples; the test is whether we can harness these observations and
thousands of others like them generated by evaluators of e-learning in VET in Europe and
elsewhere. Can we map them scientifically and record them so that other evaluators and
practitioners have easy access to them? Can we generate tools to measure their
significance? Can we identify the critical success factors not simply describe yet more
`examples of good practice’? And as evaluators can we influence the research agenda in
order to test the validity of the observations empirically and to make useful
generalisations and predictions for the benefit of future practice?

References
Alliger G M and E A Janak (1989) Kirkpatrick’s Levels of Training Criteria: Thirty years
later, Personnel Psychology (42)
American Society of Training and Development 2001
www.astd.org/virtual_community/comm_elrng_rdmap/e_lrng_comm_top_page.html
Carnevale A and Schulze E (1990) Return on Investment: Accounting for Training,
Training and Development Journal, Alexandria VA ASTD Publishing
Kaufman R, Keller J and Watkins R (1995) What Works and What Doesn’t: Evaluation
beyond Kirkpatrick, Performance and Instruction 35 (2)
Kirkpatrick D (1975) Techniques for Evaluating Training Programmes Alexandria VA
ASTD
Changing Technological Management (2000) Report by LSDA and Sheffield Hallam on
the evaluation of the Further Education National Learning Network, Sheffield UK
Learning and Skills Council.
Scrivens M
http://eval.cgu.edu
Stern E (2002) Editorial, Evaluation: International Journal of Theory, Research and
Practice, Vol 08 Issue 02, Sage UK

79
Practical evaluation interventions in understanding
informal learning within SMEs

David Slater, ??? (Ireland)

Introduction
“For many workers, perhaps most, the workplace represents the only or
most viable location to initially learn and or/develop their vocational
practice. Workplaces are becoming even more salient, as the responsibility
for maintaining the currency of vocational practice is now being
increasingly transferred to workers in the current reformulation of lifelong
learning policies and practices. In this context, opportunities to engage in
work, the kinds of tasks individuals are permitted to participate in and the
guidance provided become key bases to understand and evaluate how and
what individuals learn through their work. It is important to understand
how workplaces afford individuals or cohorts of individuals these
opportunities.” (Billett, 2001)
Billett suggests that informal learning is the primary form of vocational learning – both
initial and ongoing – for most people in the workforce today. This is presumed to be
especially true within smaller organisations, for a number of key reasons:
• There is unlikely to be a training division or visiting training unit, as one
might expect to find in a larger organisation.
• In smaller organisations, there is less likely to be a strict demarcation of
roles and responsibilities. A worker is more likely to have to ‘fill in’ for
a colleague whose job he or she is not entirely familiar with.
• Workers are often initially selected by smaller companies on the basis of
their flexibility and diversity of experience. Likewise, there is a
personality type that lends itself more to work in SMEs than larger or
State organisations.
• The ‘entrepreneurial spirit’ often pervades smaller organisations. This
encourages people to attempt tasks or activities, with the support of
others, of which they have little experience. SMEs are less likely to be
risk-adverse than larger organisations.
• Often all staff will be known to one another and the hierarchy is
relatively flat. This allows informal learning to be initiated and sustained
on an ad hoc and unstructured basis.
• The interdependencies of fate and task between staff members are more
obvious and transparent.
• The organisation, as a whole, may have to respond quickly to changing
external circumstances. These could be market, technology or
competitor driven.

80
However, as we shall discuss later these are the assumed characteristics of SMEs with
regard to their amenability to effective informal learning. Further, these are the
characteristics that many SMEs insist, when asked, apply to themselves. In some cases
some or all may apply. In others, the SME may actually exhibit qualities that make
informal learning far more problematic then the above picture would suggest.
Taking a broader view, SMEs are under increased pressure to innovate if they are to
survive. This applies not only in the ‘new’ sectors but across all fields of economic
activity. A study of 34 SMEs in the manufacturing sector in an Objective 2 region
(Atwell and Hughes, 2002) reviewed the motivators toward innovation that exert
themselves on the planning and strategic thinking processes. The definition of
‘innovation’ used in this study was:
“…any purposeful change in product, process, work organisation,
technology or work environment which is novel in a specific work
context…”
This definition of innovation, change within the individual company rather than a
technological or other advance per se, matches closely with and will be accompanied by
organisational learning and adaptability. The work identified three types of motivator:

Stimulus. A specific and particular action or event which provokes a specific and
particular response. Typically, one-off or isolated events which precipitate change.

Catalyst. The presence of a factor or factors ‘in the background’ which speed up the rate
of change. Typically these factors will be present over a period of time rather than being
one-off events.

Imperative. The ‘must-do’ situation – an event or series of events that make change
inevitable and usually urgent with identifiable negative consequences in the event of
failure to change.
Each of these motivator types may be internal or external to the organisation. The table
below provides an example of each type.
Stimulus. Catalyst. Imperative.
A new manager who Falling profits, Company going
Internal is ‘environmentally shrinking markets, bankrupt. Severe
conscious’. increasing materials Industrial Relations
waste. Problems.
A rival business Growing public Change in primary
External opens targeting the awareness of legislation on
same clients. environmental environmental
issues. issues.

While an effective response to any of these challenges will inevitably contain a learning
element, the nature of that learning and the circumstances under which it can take place
will vary greatly. For example, where the motivator is a catalyst, learning can be planned,
structured and take place over a medium or longer term. Where the company is facing an
immediate crisis, learning may be emergency and heuristic in nature.
From an evaluation perspective, the factors which define the culture, attitudes and
structures of the organisation determine the extent to which it is a suitable setting for

81
effective informal learning to take place. Where the organisation is not ‘informal learning
ready’, the role of the evaluator is to assist it to move toward that state.
The factors which motivate change and innovation have a strong bearing on the nature
and function of any informal learning that does take place. The effectiveness of the
learning will be judged, ultimately, by its capacity to meet the challenge faced.
In this paper, we will take a brief look the application of the work of Argylis when
evaluating company culture and internal interactions. We will then look at the work of
the Art Kleiner and George Roth in mapping organisational, as opposed to individual,
‘Learning Histories.’ This work was undertaken at MIT’s Centre for Organisational
Learning. What we have done here is to take this work and see how it might realistically
be applied within an SME.

Moving toward a ‘Learning Organisation’.

Before attempting to evaluate informal learning at the workplace, it is essential that the
core characteristics of an organisation that is amenable to informal learning be
understood. The nature and effectiveness of informal learning, the processes driving and
enabling it and the helping or hindering factors within an SME will reflect and be shaped
by the internal culture of that SME. Further, and with regard to evaluating such activity, it
is important to note that within micro-organisations this culture may often be an
expression of the personalities of the founding entrepreneur or entrepreneurs.
The culture of an organisation – and in particular the manner in which colleagues interact
and the practical implications of the hierarchical structure – has a very profound effect on
the processes that will accompany informal learning. This is not necessarily true for
formal learning activities, which can be ‘bought in’ from external bodies and may often
take place in a setting physically removed from the organisation itself. In such an
instance, the issue for the evaluator may often be the discovery of the extent to which
what is learned transfers, in a sustainable manner, back from the place of learning to the
place of work.
In the evaluation of informal learning, however, the evaluator must first establish the
extent to which the organisation represents fertile ground for such learning to take place.
In an action-oriented evaluation, barriers to learning should be identified and measures
put in place to address the underlying reasons those barriers are present. This has to be
done before any attempt is made to evaluate the learning process itself, or its outcomes.
Indeed, it can be argued that this must be done before any attempt at learning takes place
at all. A number of theoretical approaches, which can be directly mapped to actual
intervention, have been developed. One is described in some detail here. We can then go
on to look at the evaluation of the informal learning process. Of interest here is the work
of Chris Argylis and Donald Schon who examined the impact of organisational structures
on the behaviour of individuals within those organisations. While the scope of their work
extended far beyond looking at learning within SMEs, it is highly relevant in this context.
Argylis and Schon (1974) argue that people have mental maps with regard to how to act
in situations – including learning at the workplace with their colleagues. This mental map
determines the way that people and groups plan, implement and review their actions.
Within an SME, this mental map will be determined to an extent by external factors, such
as the society within which the SME is placed or the economic sector it operates in.
However, it is far more likely to be shaped, or at least heavily moderated, by internal
factors. In particular, it is highly dependant on the nature of the relationship between the
group and its individual members. Critically, few organisations are aware of the nature of
the maps they use – or even that they are using one at all. For example, even where an

82
organisation is convinced, or at least management is convinced, that it is has an open and
supportive learning environment the reality for intended recipients of that learning may
be at variance with this view.
For the evaluator, the identification of these contradictory perceptions of the learning
dynamic within the company – and their resolution through a consensual ‘roadmap’ for
change within the organisation - is the core task.
In exploring the nature of these relationships, Argylis developed the notion of ‘Theories
of Action’ (Argylis 1957, 1962, 1964). Smith (2001) provides a good overview of these
ideas and principles for action. In short, Argylis proposes that there is a split between
theory and action. In particular, individuals have a theory as to how they would act or
behave in a particular situation. This is referred to as their ‘espoused theory’. In many
cases within SMEs these ‘espoused theories’ may be explicitly defined through company
procedures or other documents. A number of people within the organisation – and often
people with decision making or executive authority – may also believe these theories to
be true, or at least a ‘reasonable fit’, to the reality in the office, on the shop floor or
vertically through the company hierarchy.
However, these theories often do not explain, or in conflict with, what actually happens.
This is governed by and in turn reflects the ‘theories-in-use’.
“When someone is asked how he would behave under certain
circumstances, the answer he usually gives is his espoused theory of action
for that situation. This is the theory of action to which he gives allegiance
and which, upon request, he communicates to others. However, the theory
that actually governs his actions is his theory-in-use” (Argyris and Schon,
1974: 6-7)
While theories-in-use may be more powerful predictors of actual behaviour they are often
tacit and are seldom explicitly stated. In an SME, their existence may only rarely be
recognised or their nature explored. Often they will have evolved in an organic or
unplanned fashion, and will be ‘there because they are there’. In other cases they will
reflect the relationships between individuals, units or departments within the
organisation. In a micro-organisation they may be largely derived from the personality or
approach of the founding entrepreneur.
Argylis argues that effectiveness results from developing and maintaining congruence
between the two. The challenge for the evaluator is to assist SMEs with whom they work
in recognising areas of incongruence and in identifying suitable interventions to address
those areas. Ideally, the evaluator should also be able to put in place procedures or
measures that allow the organisation to continue this work on an ongoing basis with
minimal, or ideally no, outside assistance.
As opposed to the evaluation of learning per se, this involves an audit and review of
dynamics within the organisation. Critically, the concept of double-loop learning applies
here not only to the learning process but also to the evaluation process. Double-loop
learning addresses not only the immediate learning objectives but also the governing
variables. Similarly, the evaluation has to address not only simpler ‘strategy-
consequence’ questions with regard to learning within the organisation but the ‘theories-
in-use’ that govern the interactions making informal learning effective (or even possible)
or otherwise.
Before proceeding, it is worth noting that a double loop evaluation approach presents an
inherent difficulty for the evaluator. In fact, any evaluation or audit process whose
objective is the improvement of the effectiveness of informal learning will at some time
face this dilemma. Edmondson and Moingeon (1999:60) state that:

83
 “The underlying theory, supported by years or empirical research, is that
the reasoning processes employed by individuals in organisations inhibit
the exchange of relevant information in ways that make double-loop
learning difficult – and all but impossible in situations in which much is at
stake. This creates a dilemma as these are the very organisational
situations in which double-loop learning is most needed.”
If we re-phrase this in relation to the double-loop evaluation of the suitability of a
company to effective informal learning, it suggests that the more amenable a company is
to such an evaluation approach the less likely it is to need it and the more ‘informal-
learning ready’ it actually is.
However, reality for an evaluator operating on the ground will often not be so simple.
There are a number of considerations here:
• The strategy adopted by the evaluator should tend to allow slow and
interactive discovery of the principles and practical benefits of a holistic
approach to the evaluation process. Within an SME exhibiting strong
Model I characteristics (see below) caution and patience will be
essential.
• Often, and particularly in smaller and medium enterprises, the
identification of a ‘champion’ will be critical to success. This person, or
persons, will serve to assist the evaluator in overcoming resistance to the
approach.
• External forces may be driving the process. These might include market
forces, micro or macro economic factors or the behaviour of
competitors.
These are the ‘motivators’ described above.
In any case, for the evaluation to succeed the evaluator must ensure that the process is
‘owned’ by the organisation and individuals within it. It may well be that for informal
learning to succeed within an organisation, structural or attitudinal change will be
necessary. It must be recognised that in almost all cases such change will be viewed as a
threat by some individuals who will be resistant to change.
Most usefully for evaluators in this regard, Argyris developed two models of theories-in-
use. The first, and in his experience the prevalent, is one where learning is inhibited. The
second is one where it is supported. The role of the evaluator is to assist organisations in
moving from model I to model II. It is useful here to outline the characteristics of the
theories-in-use in each model type:

Model I - Theory-in-use characteristics.

The governing values are:
• Each actor sets out to achieve the purpose as the actor sees it. In the
context of work within an SME, the extent to which this applies to
decision makers or the founding entrepreneur(s) will be of critical
importance.
• Win, do not lose.
• Suppress negative feelings. Communication is poor. This is clearly a
barrier to informal learning.
• Emphasise rationality.

84
Primary Strategies are:
• To gain control of the environment and tasks unilaterally.
• Protect self and others unilaterally.
Operationalised by:
• Unillustrated attributions and evaluations.
• Advocating courses of action which discourage inquiry.
• Considering ones own views to be obviously correct without any
apparent reflection.
• Making covert attributions and evaluations. Often this will result in a
climate of rumour within the organisation.
• Face-saving moves such as leaving potentially embarrassing facts
unstated.
Consequences include:
• Defensive relationships.
• Low freedom of choice.
• Reduced production of valid information.
• Crucially, there will be little public testing of ideas.
A model I organisation is predominantly characterised by defensiveness and
protectiveness. Model II organisations are characterised by a very different set of
theories-in-use:

Model II - Theory-in-use characteristics

The governing values of Model II are:
• Valid information.
• Free and informed choice.
• Internal commitment.

Strategies include:
• Sharing control.
• Participation in the design and implementation of the action.

Operationalised by:
• Attribution and evaluation illustrated with relatively direct
observable data.
• Conflicting views will surface.
• Encouraging the public testing of evaluations.

Consequences should include:

• Minimally defensive relationships.
• High freedom of choice.
• Increased likelihood of double-loop learning.

Argylis points to an increased likelihood of double loop learning as a possible

consequence of an organisation moving from Model I to Model II. In reality the
consequences for an SME, particularly one operating in an innovative sector or
approaching traditional economic activities in an innovative manner are deep-reaching

85
and will affect almost every area of the company’s operations. Such an organisation also
exhibits the characteristics most likely to make informal learning work. This may occur
as an indirect impact of its increased capacity to introduce innovative working methods.
Indeed, it can be argued that moving an organisation toward Model II will have the
inevitable, even where unintended or unplanned, outcome of greatly enhancing the
informal learning process.
The role of the evaluator is to help the organisation move from Model I to Model II – or
at least as far toward those elements of Model II that are most closely implicated in
assisting effective informal learning as practicalities, time and the organisational culture
allow. It should be noted that it is not anticipated that all aspects of the organisation
should conform to the Model II type - or that it would be the goal of the evaluator in this
work. The nature of commercial organisations within capitalist economic systems will
produce inherent hierarchies of power and control. Organisations are political and it is
important to recognise this.
The key elements of Model II with regard to enhanced informal learning are:
• Internal Commitment.
• Participation in the design and implementation of action.
• Encouraging public testing of new ideas.
• Minimally defensive relationships.
The objective of the evaluation is to identify the main ‘theories-in-use’ that apply in each
case and to explore their levels of congruence with espoused theory. Interventions must
then be put in place to change the default theories-in-use. The work follows six broad
phases. At all stages, the criteria specified in the Model II description should serve as the
core indicators of progress.

Phase 1. Understanding the current situation.

In this phase of the work, the current theories-in-use and espoused theories are explored.
The word ‘situation’ rather than ‘problem’ is used carefully here as this work must refer
to more than specific recurring instances where difficulties are encountered.
This phase of the work will involve intensive consultation with individuals from the
organisation. In some cases, this may involve one-to-one interviews with key personnel -
for example, the Human Resources manager or founding entrepreneur(s). It may also
involve focus groups or other information sources such as questionnaires. The emphasis
in this work is to identify the theories-in-use that dominate, and where possible to trace
their histories and origins.
This work may also involve looking at other aspects of the organisation such as its
structure, operational procedures, ownership or staff benefits. This will give a clearer
picture of the SMEs espoused theory. As noted above, this will often be explicitly
available from documentary or other sources.
On the basis of this, a first outline or ‘map’ of the learning dynamic within the
organisation can be drawn up.

Phase 2: Agreeing the ‘Map’ with clients.

The next step, which will to an extent overlap both chronologically and in tasks with the
first, is to agree the ‘map’ with the client. This is the most difficult, sensitive and
important phase of the work. Initial conclusions can be presented back to the client. As
the nature of this work may be less confidential than work on the first phase larger and

86
more ‘mixed’ – in terms of roles and responsibilities within the organisation – groups
should be involved.
The objective here is not just to reach agreement with the client, but to ensure that the
‘map’, its origins and impact on the operation of the organisation and the working lives of
individuals within it are fully accepted and internalised. This may take some time, and
require considerable honing of the initial ‘map’.
This phase will determine the overall success of the initiative. It should serve in itself as a
mini pilot of Model II behaviour within the organisation. Discussions should be frank and
open, people should be unafraid to speak their minds and responsibility for previous
mistakes or misunderstandings should be taken. This phase, while ostensibly serving to
agree the ‘map’ of the current situation, is a core element of the learning process itself.
Also, it should be undertaken in such a way as to allow ‘ownership’ of the process to pass
into the hands of people working within the organisation.
As part of this process, objectives should be identified where possible. These could be
broad or narrow in focus, short or long term in duration. Setting objectives is important
for two main reasons. It serves to make concrete the discussions and remove any
impression that they may be strictly theoretical or exploratory. By doing so, it may help
to reveal facets of the real situation that are pertinent in this work. Secondly, it provides a
framework for the later joint evaluation of the outcome and future needs.

Phase 3. Validate the agreed ‘Map’ and identify Interventions.

Once the ‘map’ has been agreed it can be verified by deriving testable predictions. These
can be done by looking to practice and history in conjunction with members of the
organisation or by communally asking ‘what if’ questions. Methods to use here might
include focus groups or role playing sessions.
If the map appears to properly reflect the reality of the organisations culture and activity
the evaluator can begin to formulate interventions. Much of the ‘what if’ work will
already have begun this task. Examples of interventions might be:
• Changing the physical layout of the workplace to facilitate cross-
communication between different units or workgroups.
• Introducing a ‘mentor’ system, where more experienced staff would
work in a structured fashion with others.
• Making paid time available each week for workers to explore their own
learning needs. For example, the company might agree to make other
staff available to them during this period if those staff could meet
particular learning needs.
• Establish a ‘planning learning group’ drawn from all departments to plan
future developments.
These examples chosen at random illustrate the scope of possible interventions. It is also
important to note that informal learning can take place in a structured manner and actions
can reflect this. Some of these examples may not at first glance appear specifically
learning-related at all. Through discussion, the most appropriate interventions are
selected. These may be selected on the basis of:
• Match to objectives agreed during Phase 2.
• Match to overall organisational objectives.
• Match to other activities or initiatives planned.

87
 • Acceptance. Is the initiative broadly accepted across the different
elements of the organisation?
• Benefit. Expected benefit of the intervention.
• Simplicity. At first, easier interventions may be more appropriate. Not
only does this allow for gradual development of informal learning
capacity, it may also facilitate the evaluation process.
• Scope of involvement. The extent to which the intervention involves a
broad range of individuals, functions and departments.
• The expected capacity of the intervention to illuminate the internal
processes helping or hindering its introduction. To what extent does it
touch on the more sensitive areas of the agreed ‘map’.
• Cost, both in terms of direct investment and any labour hours required.
The decision as to which intervention(s) are to be implemented should be taken in as co-
operative a fashion as possible.

Phase 4: Implementation of Identified Intervention(s).

There can be no prescriptive outline of this phase of the work. It will be different in each
instance, dependant on the nature of the intervention and that of the organisation.
However, a number of factors should be borne in mind during this phase:
• It must be agreed in advance that sufficient resources for full
implementation will be available.
• The evaluator should pass as much control over to participants as is
feasible and technical and management considerations allow.

• Where a small team or sub-group is involved this should be as internally

democratic and representative as possible. It will still, of course, be
answerable to and part of company structure.

• Regular review should form part of the intervention. This should allow
for ongoing change. It should also serve to act as information and
feedback for a later full review.

• The intervention should be introduced on the basis of an understanding

that if it does not meet the originally stated objectives, or if those
objectives change or become obsolete or less relevant, it will be
discontinued. When this happens, resources will be passed to other or
new interventions.

Phase 5: Review, Evaluation and Future Planning.

In evaluating the effectiveness of the informal learning initiative, a dialogical evaluation
approach would appear to be the most appropriate in the majority of cases. This is partly
because the process that led to the introduction of the intervention should have been
dialogical in nature, but also because it reflects the nature of a Model II organisation. If
there are difficulties in conducting a relatively open, reflective and interactive review,
this in itself is indicative of the success or otherwise of the project.
The role of the evaluator, then, is to facilitate this dialogue and to draw out conclusions
and recommendations. However, there are a number of aspects to the work that the
review should address (Jeffs and Smith, 1999).

88
Interactions. What were the characteristics of these? To what extent were they educative
or informative? How were they initiated and were they formally or informally initiated?
Have they been sustained? If not, why not? Do they reflect the sort of values we are
trying to encourage?
Focus. What issues and topics formed the core of the informal learning that took place?
Was this what was originally planned? If not, why not and was the actual focus more
appropriate? Was what was learnt applicable in the workplace? Was this directly
applicable in peoples work or did it serve more to change the nature of their interaction
and relationship with the organisation?
Setting. Was the setting appropriate? If a variety of settings were tried, which was most
effective? Did this vary with the nature of the interactions and focus?
Aims. What were the aims from the outset? In hindsight, were they realistic and
achievable? Did they change and if so why? Did conflicts of interest arise with regard to
aims and objectives? How were they resolved?
Resources. Were the resources sufficient? If not, why not? Was there considerable
overspend, and if so why? If more resources were needed, were they forthcoming?
Strategies. How did our original learning strategies change? How, if at all, were they
altered and who altered them?
Outcomes. What were the outcomes for different participants? What were the direct
outcomes for the organisation? Were there any unexpected or unintended outcomes?
Were there any negative outcomes?
Next Steps. What main lessons have we learned? What are the key barriers to moving
toward a situation where informal learning might be more successful? What is needed to
move informal learning in the organisation toward Model II? Who needs to be involved?
What structures need to be in place? Who needs to know about all this?

Learning Histories - Turning Experience into Action.

Even where an initial pilot or extended pilot to set in place the structures and interactions
needed to ensure effective informal learning is successful, measures must be taken to
ensure the sustainability of the intervention. Further, when an organisation undergoes
change as a result of one of the innovation ‘motivators’ described in the introduction to
this paper, it is important that lessons can be learned from those events and that those
lessons are gathered, presented and disseminated in such a way as to influence future
behaviour. A mechanism for the on-going monitoring, feedback and correction of internal
processes is required. Ideally, this should be one that requires little external support from
external evaluators or experts.
The ‘Learning History Research Project’ at MIT developed a set of tools and techniques
to address these and other issues. In particular, the project recognised that within
‘learning organisations’ a number of key information, evaluation and assessment needs
should be addressed (MIT Centre for Organisational Learning, 1997). These are:
• The leaders or champions of all learning and change efforts will
eventually have to internally justify their accomplishments and show
that their work represents a valid investment.

89
 • Companies need to know the value of their learning experiences to date.
They also need to understand how successes in one part of the company
can be replicated in others.
• Participants need to be able to judge the value of their past experiences
in informal learning and make a concrete input into future plans or
actions.
• It should be possible to assess past actions and provide feedback to
ongoing processes in a way that does not encourage a reversion to
defensive, Model I type, behaviour.
• To properly plan future actions, people within the organisation should be
able to understand past events from other people’s perspectives.
The learning history is a structured process, with structured outputs, which address these
issues. A Learning History (Kleiner and Roth, 1997) is described as:
“A ‘learning history’ is a document, or a series of documents, possibly in
audiovisual or other format, that is disseminated in a deliberately
structured manner. The document, and the dissemination, are both
designed to help the organisations to become better aware of their own
learning and change efforts.
The learning history presents the experiences and understandings of
participants, people who initiated, implemented and participated in
organisational transformation efforts, or some collaborative learning
experience, as well as non-participants who were affected by those efforts.
The learning history tells the story in the participants own words and in a
way that helps the rest of the organisation move forward without having to
re-invent what a small group of learners have already discovered.
A learning history thus represents the organisation talking to itself, in a
safe and carefully structured way, about the things it needs to hear but
hasn’t yet listened to.”
Much of the work of Kleiner and Roth looked at the implementation of change within
organisations and how lessons from such processes could be learned. A rigorous
procedure for implementation of the method has been developed. In this context, we
might look at using parts of this method in order to provide ongoing review of informal
learning within an SME. In particular, we will look at how this might be done with
minimal external support, by people within the SME.
We assume here that the SME has undertaken the groundwork as described above in
order to position itself as a ‘learning organisation’ or at least one where informal learning
is supported and its importance recognised. Intensive work has taken place to identify the
theories-in-use and espoused theories, which govern behaviour and this map has been
agreed and tested. Interventions have been put in place to improve informal learning and
exchange of ideas. Some of these may have been done on a pilot basis, and it is important
to extend and continue the successful trials. On a continuous or intermittent basis,
external or internal motivators are moderating the circumstances under which this
learning is taking place.
Essentially, the ‘Learning History’ approach supports reflection and the exchange of
opinion. Interviews take place with all those directly concerned. This would be an
onerous and very expensive task within a large organisation but can be managed by a
relatively small team within an SME. Critically, this team would use, and be trained to
use, the ‘jointly-told’ story technique that lies at the heart of the learning histories
method.

90
The history is separated into ‘short stories’, each with a particular focus. For example,
these might refer to a particular learning initiative, internal project or production run.
Each of these is presented in a three-part format. First, the known, agreed or observable
facts are presented. This section, essentially an introduction, is given using the whole
width of the page under a header describing the episode or event being described.
The page is then split into two columns. On the right hand side is given transcripts or
excerpts of transcripts from the relevant interviews undertaken. Roles, but no names, are
shown. On the left comments or opinions from the ‘learning historians’ are shown. In the
original MIT schema, this would be a mixed team drawn from outside experts and key
company personnel such as the Human Resources Manager or the Head of Training.
These would have an overview of all documentary evidence and together would build up
the story line. It is unlikely that this approach would be effective in an SME, partly due to
the expense of retaining such persons on an ongoing basis. A small team would be
assembled to do the work on an occasional basis – or in response to a particular event or
situation – with some access to outside support if needed.
In addition to these elements of the document, full width interludes are included are
added as required to describe background events or a change in the focus of the
interviews, for example that we are now moving to a different department or
geographical site. Here is an example of a ‘jointly-told story’ page. The content is purely
illustrative.

Decision to house graphics and software design teams together.

As a result of the last review, it was decided to move the software and graphic
design elements of the web team so that they shared the same room. This was
done because it was generally felt that communications between the two were
poor, which was affecting project delivery. Also, an ‘us and them’ culture was
developing between the two teams – even to the extent that social relations
were beginning to break down.
It was felt that much of the antipathy existed because each team had no real
grasp of what the other was doing, or the difficulties they faced.
A mistake appears to have Graphic Designer : Strangely, although I
been made here in that the two felt that the move was beneficial to both
groups were not consulted in teams I must admit that I still resent the way
full and together before the that it was done. There was little
move. Only the leaders were consultation, it was presented to us as a fait
consulted, who felt they were accompli. I have learned more about the
fighting their corner in the software side of the development – but to be
discussions. honest I enjoy my work less now.
Head of Software Team : This move has not
This is a difficult issue to been beneficial for us because the graphics
resolve. It would appear that team have a different way of working. For a
the two working together does start, they make a lot more noise and talk a
present operational problems lot more than we are used to. Not banter, it’s
but has solved many of the work related talking but it just appears that
deeper seated issues that were they have to communicate a lot more as they
destroying cohesion in the go about their business. I have learned a lot –
company as a whole. and in particular about what they need from
us in terms of requirements – but
productivity has definitely fallen.
After these interviews were conducted, it was decided that the weekly

91
 production meetings would take place in the room and would involve all the
members of both teams rather than just the group leaders.

Using this technique, the learning historian narrates and annotates events and processes
within the SME. A number of core phases would be involved in this process.

Phase 1 : Planning and Scope.

It is not expected that this activity would be constantly ongoing, but would be conducted
occasionally in response to specific events or as part of planned, regular review of
interventions or actions within the SME. Before proceeding, the areas that need to be
addressed need to be identified – the ‘titles of the episodes’. This may entail an initial
consultative process to agree the focus of the work.

Phase 2 : Interviews and Data Gathering.

The ‘learning historian’ should gather the perspective of every significantly involved or
affected person. These interviews should encourage the interviewee to be reflective about
events and be open to consideration of future actions. The interviews are both an input to
and an outcome of the process. Their reflective nature should give the interviewee space
to think about broader issues within the organisation in a way that he or she might not
normally have time to.

Phase 3 : Identifying Key Issues.

When the interviews have been completed, the information collected is gathered and
emerging lessons collated. These may relate to the individual themes or areas addressed
or may relate to the company in a more holistic manner.
In this phase of the work the services of an outside person, familiar with the process and
preferably working with the SME in this role on an ongoing basis, may be used. With the
support of that person where required, the ‘learning history’ document is drawn up.

Phase 4 : Participant Consent.

Once the ‘learning history’ has been completed participants are shown the extracts
relating to their interviews, and the attached comments, before they are circulated to any
other person. This is critical to maintaining trust in the process.

Phase 5 : Reflective Feedback.

In workshops, people from every part of the organisation, having read and studied the
document, discuss its implications. How accurate was the document? Do people now
have things they wish to add, or take away, from it? What are the next steps forward?
What has been learned at the level of the SME as a whole? What was successful or
otherwise about the whole process?
Ideally, the ‘learning history’, which creates an ongoing narrative of the culture and
dynamic within the organisation, will serve as a powerful instrument for understanding
the nature of interactions, including informal learning, within it.

Conclusions.
Most SMEs recognise the need to improve communications and informal learning
processes internally. While they would aspire to becoming ‘learning organisations’ there
are formidable barriers in their way. Not least may be current practices which hinder such
progress but go unrecognised and unchallenged.

92
To maximise the effectiveness of their internal and informal learning processes, SMEs
may first need to:
• Identify those barriers currently in place. It is suggested here that this be
done through exploring the ‘theories-in-use’ and ‘espoused theories’
currently governing behaviour. However, other methods may be more
appropriate depending on circumstances.
• Explore interventions that will serve to overcome those barriers. These
should be reviewed on an ongoing basis and moderated as appropriate.
• Put in place mechanisms for the ongoing review of informal learning
within the SME.
It is recognised that an SME cannot be normally expected to spend considerable time or
resources on these activities, save as a response to recognised and acute difficulties.
However, where relatively simple mechanisms can be put in place, requiring the
minimum of external support, SMEs can considerably boost the potential for informal
learning to take place among their staff.

References
Anderson, L (1997) Argyris and Schons theory on congruence and learning. [On Line].
Available at http://www.scu.edu.au/schools/sawd/arr/argyris.html
Argyris, C. (1957) Personality and Organisation, New York: Harper Collins.
Argyris, C. (1962) Interpersonal Competence and Organisational Effectiveness, Home
wood III.: Dorsey Press.
Argyris, C. (1964) Integrating the individual and the Organisation, New York: Wiley
Argyris, C. (1965) Organisation and Innovation, Homewood, III.: R.D. Irwin.
Argyris, C. and Schon, D (1996) Organisational Learning II : Theory, method and
practice, Reading, Mass: Addison Wesley.
Argyris, C. and Schon, D. (1974) Theory in Practice : Increased Professional
Effectiveness, San Francisco: Josey-Bass
Billett, S. (2001) Participation and continuity at work : A critique of current workplace
learning discourses. Context, Power and perspective : Confronting the Challenges to
Improving Attainment in Learning at Work. Joint Network / SKOPE / TRLP
International workshop 8-10th November 2001, Sunley Management Centre, University
College of Northhampton. Available at the informal education archives:
http://www.infed.org/archives/e-texts/billett_workplace_learning
Jeffs, T. and Smith, Mark K. (1999) Informal Education. Conversation, democracy and
learning, Ticknall: Education Now Books.
Kleiner, A. and Roth G. (1997) Learning Histories. A New Tool for Turning
Organisational Experience into Action. http://ccs.mit.edu/LH/21wcp002.html
MIT Centre for Organisational Learning. (1997) Field Manual for the Learning
Historian. http://www.ccs.mit.edu/LH/intro.html
Smith, Mark K. (2001). Chris Argylis: Theories of Action, double loop learning and
organisational

93
Evaluating the effectiveness of e-Learning Strategies
in SMEs

Eduardo Figuiera, Academus (Portugal)

IWhat is e-learning?
How individuals live and work has suffered significant changes lately due to innovations
introduced to and carried on by the information and communication technologies (ICT).
In fact, ICT is the basis for the society of today and, in addition, it is “helping” the
society to move rapidly towards a Society of Knowledge. In this context, e-learning will
play a particular and relevant role for individuals once it will give more power to them in
terms of choosing their learning preferences. E-learning strategies will not only allow
individuals to choose the contents, time and pace for learning but also produce a true
revolution in the education and training systems. This is very important because learning
plays a central role in knowledge creation which by its turn is essential for making the
transition from Society of Information to Society of Knowledge. However, given the
relevant role that ICT plays for that transition, it is also important to understand how
different e-learning is from learning. In other words, e-learning has to be defined.
In general and simple terms, e-learning may be defined as learning opportunities
delivered and facilitated by electronic means. That means that what qualifies the e-
learning mode of learning is the fact of the learning contents being delivered by
electronic technology. However, in this line of reasoning, two questions are in order:
1) – Is the means of delivering (electronically in that case) important enough to qualify a
mode of learning?
2) - Can training/teaching contents delivered by electronic means be considered an e-
learning strategy?
According to one adult educator working in community development, e-learning
strategies are like cars; they are just a means to get faster the knowledge (place) you want
to get. So, that adult educator assumes that e-learning differs from “traditional” learning
only by the way learning contents are delivered. This may be not true. The reason for that
is because people approaches learning from different ways and perspectives.
Traditionally, people approaches learning through a third party since they usually need a
guide, a tutor, or a facilitator to learn more effectively. Only people who are mature may
not need such a third party. However, e-learning may introduce a new strategy for
approaching learning since it integrates the three needed components for learning: (1)
contents, (2) strategies, and (3) tutoring. In addition, people learn in different ways, that
is, individuals own specific learning styles which may vary across contexts, tasks and
age. So, it may be possible that e-learning makes individuals to create a new learning
style and may help academics to understand better the learning process. In fact, e-
learning strategies may introduce changes in the way people learn due to fundamentally
the increasing effectiveness of interoperability and security, (Rosenberg, 2001),
appropriateness of time and pace, opportunity and rapidness.
It is important to note that a general idea exists that e-learning corresponds to all
educational and training sets that uses ICT, including distance education. This is not
correct once the term e-learning indicates that the emphasis is on learning not on
teaching, one of the main characteristics of the education systems. That does not mean,

94
however, that e-learning should be goal free and disconnected to an organized system
being conducted completely on the learner’s responsibility. What means is that the e-
learning strategies should have individual’s learning as the system basis and its main aim
and assume that responsibility for learning must gradually be transferred from a tutor or
facilitator (a third party) to learner in function of his/her level of maturity. In addition, e-
learning strategies should also integrate a system for individual’s assessment based on the
process of assessing learning outcomes. Assessment of outcomes constitutes a more
effective and rigorous indicator for individual’s learning than the traditional assessment
still conducted by many educators in the education and training systems. Furthermore,
learning does not happen in an empty space or outside time, without definition of goals.
Learning is contextual since it is influenced by history and memory, by cultural and
community identity and values (Magli, 2003).
Unlike many of web-based ideas for businesses and other activities, e-learning has got
through the economic crisis of the two last years. However, before e-learning reaches the
fullest potential, it needs to go through three other changes (Reeves, 2003). First of all, a
mental model for e-learning has to be developed and expanded in individuals’ minds.
Mental models are cognitive structures built by individuals’ minds which influence the
way they react to change and solve problems (Reeves, 2003). Like learning styles,
usually, individuals construct different mental models in function of the experience they
have with the object about which they construct the mental model. For this reason, people
related to marketing of e-learning products often describe e-learning as media events
using terms such as “full motion video” and “dynamic learning objects” (Reeves, 2003).
Instructional programmers and trainers probably see e-learning as training activities
analogous to “presentations” and “group discussions”, for example (Reeves, 2003).
Subject matter specialists, by contrast, conceptualize e-learning as an alternative
(electronic) strategy for delivering content. Second, quality of assessment within the e-
learning products must be improved. Outcomes show if and how much learning has
occurred, performance has changed, and results have been attained (Reeves, 2003). In
fact, assessment of outcomes has been considered essential for education and training,
reflecting the emphasis on learning followed lately by most educators (Huba & Freed,
1999; Marzano, Pickering, & McTigh, 1993, cited by Reeves, 2003). This is much more
evident in the business world once what is important is performances being correlated
and connected to company’s goals.
And third, rigorous evaluation strategies of e-learning initiatives should be taken as a
serious business (Reeves, 2003). This is important because everyone needs to know if
outcomes have been achieved, i.e., certification of outcomes has a very important role to
play in e-learning.
The aim of this paper is to provide a framework to measure effectiveness of e-learning
strategies or programmes. This will be done without referring to any particular e-learning
programme or strategy.

Evaluating the effectiveness e-learning as a strategy

to see its worth
Development and use of any e-learning programme or strategy represents an individual,
organisational and social investment. For this reason, effectiveness of any e-learning
offering should be evaluated. Without knowing efficacy of e-learning strategies one
cannot know if it is worth of using them or not. So, measuring effectiveness can
constitute a very useful tool to base one’s decision about use of any e-learning strategy.
In addition, built-in programme evaluation allows trainers and other formative

95
responsible to monitor e-learning programmes and offerings and make changes for
improvement. In fact, given the potential for changing of the e-learning strategy,
mandatory evaluation represents a relevant characteristic since it permits to know how
much worth for people and organisations are the e-learning offerings.
Globally, programme effectiveness can be evaluated following five types of evaluation
approaches (Table 1): (1) Based on the programme goals, (2) Based on the decision-
making process, (3) Goal-free approach, (4) Based on an expert’s knowledge, and (5)
Naturalistic approach (Pietro, 1983).
Evaluation approach based on the programme goals is fundamentally oriented for
verification and quantification of execution of the programme goals. For this purpose,
experimental or quasi-experimental research models are used in a 4-step approach: (1)
Identification of the programme goals, (2) Translation of the programme goals into
quantitative data, (3) Data collection from individuals related to the programme, and (4)
Comparing established goals with reached goals.
Evaluation approach based on the decision-making process takes this process as the main
element of the evaluation model. Although theoretically this approach appears to be of
simple execution, in practice its implementation is less simple than expected due to the
complex nature of the decision-making process. Assuming that reality and taken
evaluation as a process of designing, collecting, and furnishing useful information for
judging alternative solutions, Stufflebeam has proposed the CIPP approach (Context,
Input, Process, Product) (Stufflebeam et al., 1971). The CIPP approach is based on the
identification of five programme aspects (Pietro, 1983): (1) the decision-making process,
(2) the scenarios in which the decision-making process occurs , (3) the appropriated
decision-making models to each scenario, (4) the basic types of decision that are taken,
and (5) evaluation types that best fit each one of the decisions.
The goal-free evaluation approach is based on the assumption that evaluation should be
oriented to assess not only the established goals but all consequences that a programme
has produced.
The evaluation by an expert approach is an evaluation conducted by an individual or a
team specialised on the object being evaluated. The basic principle of this approach is
based on the human judging.
The naturalistic approach is based on the assumption that there rarely exist a true that is
accept by all people involved in a programme (Pietro, 1983). For this reason, the
naturalistic evaluator try to respond to the major number possible of questions posed by
the all individuals involved in the programme.

96
 Evaluation Approaches

MODEL MAIN AIM TYPICAL QUESTIONS METODOLOGY

Have goals been reached?

Quantitative.
Estimating how goals Have they been effective
Programme Goals Experimental design.
have been reached and efficiently reached?
Before and after testing.
Were goals appropriate?

Estimating value based on

hypothesis related to
Is the programme effective?
Context, Input, Process,
Should the programme be
Furnishing information and Product.
Decision-making continued, ended, or
modified? How? Participation of decision
makers concerned with
value of the programme.

What are the intentional and

Independent estimation
not intentional programme
Estimating efforts from needs and judging
results?
Goal-free developed by the criteria.
programme
How much value have the Quantitative and
programme results? qualitative techniques.
Critical revision based on
Experts as judges How an external Professional experience and collection
Expert
and evaluation tools classifies the programme? of specific information and
subjective opinions.

What is happening to
programme? Inductive discover based
on qualitative methods.
Understanding process What are the different
Naturalistic perspectives?
underneath programme Open interviews,
How is programming participant observation
responding to the different and case studies.
interests?
Font: Pietro, D.S. (1983). Evaluation Sourcebook for Private and Voluntary Organizations.
ACVAFS (American Council of Voluntary Agencies for Foreign Service), New York.

A Framework for Evaluating Effectiveness of e-

Learning strategies
The Proposed Global Approach
The global approach proposed to evaluate effectiveness of e-learning strategies can be
considered as a mix of the decision-making, goal-free, and expert evaluation models.
More specifically, the evaluation approach is framed by Bennett (1979)’s system of
criteria for measuring programme impacts (Fig. 1). According to Bennett’s systems of
criteria, to one hierarchy of impacts correspond one hierarchy of evidences. That
hierarchy of evidences comprehend the following aspects: (1) Human and financing
resources; (2) Developed activities; (3) Participation; (4) Reactions; (5) Change of
knowledge, attitudes, abilities, skills and aspirations; (6) Change of practices; and (7)
Final results.

97
Since this framework intends to be utilized for measuring effectiveness of e-learning
strategies, it was considered appropriate to aggregate into three the 3,4, 5, 6 and 7 of
Bennett’s hierarchy:
Participation (level 3);
Reactions (level 4);
Results (levels 5,6, and 7).

Figure 1: Bennett’s hierarchy of evidences
7. Final Results
6. Change in
practice

5. Change in knowledge, attitudes,

skills aspirations

4. Reactions

3. Participation

2. Activities

1. Resources

In addition to aggregation of the 3,4,5,6, and 7 Bennett’s levels, participation is

conceptualized as a set of the stakeholders’ interventions in each phase, aspect, and
activity of the e-learning programme. Similarly, reactions are taken as the target
individuals’ responses to each phase, aspect and activity of the e-learning programme.
Taking those changes into consideration, the Bennett’s hierarchy applied to evaluating
effectiveness of e-learning strategies can be transformed in a 4-level hierarchy of
evidences: (1) Resources, (2) Participation, (3), Reactions, and (4) Results (in terms of
acquired knowledge, skills, and behaviour) (Fig.2).

Figure 2: Adapted from Bennett’s hierarchy of evidences

4. Results

3. Reactions

2. Participation

1. Resources

98
Criteria to evaluate effectiveness of e-learning
In addition to the global approach above presented which essentially serves as a
conceptual matrix of reference to guide development of criteria and indicators, measuring
effectiveness of e-learning needs a set of appropriated criteria. With that reference in
mind, it appears that measuring effectiveness of e-learning should take into consideration
nature, characteristics and objectives of the e-learning strategies. For this reason,
approaching evaluation of e-learning effectiveness may be following e-learning
dimensions: (1) Organisational & Management dimensions, (2) Pedagogical dimension,
(3) Technological dimension, (4) Ethical considerations, (5) Learning assessment &
certification, and (6) Evaluation Strategy.

a) Organisational & Management dimensions
This dimension integrates aspects concerning organisational and management issues and
how different types of affairs and services are organised and managed. It takes into
consideration questions such as:
♣ How administrative affairs are organised and managed?
♣ How learning related affairs are set up and functioning?
♣ How help services are organised and offered?
♣ How information is organised, managed and distributed?
♣ How learning environment is designed, set up and managed?
♣ How instructional and technical support services are organised and offered?
♣ How career counselling services are set up and functioning?
♣ Is there any other online and/or offline support services? How are they
functioning?
♣ How assessment of learning is set up and managed?
♣ How certification of learning is established and functioning?

b) Pedagogical dimension
The focus of the pedagogical dimension is on aspects and issues concerning learning
needs, objectives, strategies, and environments. Those aspects and issues can be
illustrated through questions such as:
♣ Have learning needs identification and analysis been done?
♣ How learning objectives are set up ?
♣ Are learning objectives in line with SMEs’ goals?
♣ How contents are developed, organised and offered?
♣ How learning objectives are related to contents?
♣ How learning strategies are designed and implemented?
♣ How learning environments are designed?
♣ How learning environments are linked to each other?

c) Technological dimension
The technological dimension is related to aspects and issues dealing with design of the
technological infrastructure and friendliness of navigation. The following questions
illustrate what aspects are comprehended in this dimension:
♣ How technological infrastructure is designed and implemented?
♣ How learning environments are designed and constructed?

99
 ♣ How software is designed and working?
♣ How site is designed and appealing to?
♣ Is navigation ease and friendly?
How usable and functional is the web-site?

d) Ethical considerations
Ethical issues may be the first aspects that should be taken into consideration when one is
measuring effectiveness of a e-learning programme. Within the frame of this dimension
may be included questions such as:
♣ How e-learning is related to social & cultural diversity?
♣ How e-learning is concerned with geographical diversity?
♣ How e-learning is approaching learners’ diversity?
♣ How e-learning is approaching social values?
♣ How e-learning is treating legal issues?

e) Learning assessment & certification
Learning assessment & certification integrates aspects concerning essentially how
learning is assessed and certified. Those aspects assume particular importance once gives
confidence not only to participants but also to who have to work with them. It takes into
consideration questions such as:
♣ How learning is assessed?
♣ How learning is certified?
♣ Is learning assessment & certification taking into consideration learners’
diversity? How?
♣ Is learning certification officially recognized?

f) Evaluation Strategy
Such as assessing and certification of learning is fundamental for e-learners, global
programme evaluation should be considered a relevant dimension given the need to know
quality of the programming. This dimension includes questions such as:
♣ How evaluation strategy is designed & planned?
♣ How evaluation of learning environments is designed and planned?
♣ How learning environment (s) is (are) evaluated?
♣ How useful is the learning content to learners?
♣ Are impact evaluation of learning strategies designed and implemented?

Final considerations
The starting point for this paper was that one cannot know if it is worth using e-learning
strategies if their efficacy is not known. Measuring the effectiveness of elearning
constitutes a useful tool for taking decision about use of any e-learning strategy. In
addition, knowing the effectiveness of e-learning strategies allows formative monitoring
and improvement in implementation. Given the potential for change, evaluating the
effectiveness of the e-learning strategies is integral to their implementation and
improvement since it indicates the value for individuals and organisations of e-learning
provision.

100
In addition to the conceptual matrix of reference presented in the paper, which serves as a
guide for developing criteria and indicators, measuring the effectiveness of e-learning
should be based on a set of appropriate and specific criteria that take into consideration
the nature, characteristics and objectives of the e-learning strategies.

References
Bennett, C. (1977), Analyzing impacts of Extension Programs, Washington DC: USA
Department of Agriculture.
Huba, M.E. & Freed, J. (1999). Learner-centered assessment on college campuses:
Shifting the focus from teaching to learning. Needham Heights, MA: Allyn & Bacon.
Magli, R. (2003). E-learning Lost in Time and Space? The Conservative and Incomplete
Emerging e-Learning Concept. (retrieved April 20, 2003, from
http://www.elearningeuropa. info/docPrint.php?id=1289&1ng=1)
Pietro, D. S. (1983), Evaluation Sourcebook, for private and voluntary organizations,
New York: ACVAFS (American Council of Volutary Agencies for Foreign Service).
Reeves, T.& Aggen, W. (2003). Enhancing E-Learning Assessment and Evaluation
Strategies. (retrieved May 23, 2003, from www. learnwright.com/ ELearn% 202002%
20Reeves% 20Aggen% 20Paper.pdf)
Rosenberg, M. J. (2001). E-Learning: Strategies for delivering knowledge in the digital
age. New York: MacGraw-Hill.
Stufflebeam, D. L. et al. (1971) Educational Evaluation and Decision-Making, USA: F.
E. Peacock Publishers Inc.

101
E-learning challenges in Austrian SMEs

Klaus Reich and Freidrich Scheuermann, ??? (Austria)

Introduction
E-learning is changing the way enterprises gain competitive advantage through improved
human performance. Especially small and medium-sized enterprises have to face the
problem that e-learning technologies, methods and strategies have mostly been developed
for the needs of large enterprises and cannot be exactly transferred to their needs. SME’s
operate in almost every sector of the economy. As a consequence they vary widely in
their learning and training needs. They have to deal with limited personnel, organisational
and financial resources. The situation is furthermore stimulated by the difficulty to
formulate detailed training strategies that will enable their employees to be better
qualified to cope with increased competition. As they are more and more discovering the
advantages of ICT-based learning it is necessary to provide a framework that takes into
account these limitations and tries to find effective solutions. Based on a survey about e-
learning in Austrian SME’s (Scheuermann & Reich, 2002) the current state of ICT based
learning in Austria is analyzed and suggestions for better implementation and use of e-
learning are made with a special focus on cooperative e-learning approaches.

E-learning in Austrian SME’s

The average annual growth of the e-learning market in Austria is currently estimated to
be 102% compared with the European average of 96%. In 2003 this will result in a
market of 116 million Euro growing to 196 million Euro in 2004. At present, by far the
largest market share is e-learning for large enterprises, but small and medium sized
enterprises (SMEs) are increasingly discovering the potential of e-learning (Zugmann,
2001). As the results of a survey carried out for CEDEFOP (European Centre for the
Development of Vocational Training) (Attwell, forthcoming) show, the implementation
of e-learning in SMEs is dependent on overcoming a series of barriers and problems. This
articles gives an overview of the results of the CEDEFOP study on the use of e-learning
in SMEs in different European regions. It focuses in particular on the problems in the
implementation and use of e-learning in SMEs, on different pedagogical approaches, on
partners and networks and on further needs in research and development.
The Austrian case study for the Cedefop project was the Tyrol region. Most enterprises in
this area currently make no use of e-learning and only a minority are considering the
introduction of e-learning in the future (Scheuermann & Reich 2002).
Hawke (2000) identifies multiple benefits in investment in the workforce, including
increasing the employability and earnings of employees and increasing the productivity
and profitability of businesses. The minority of Tyrolean SMEs that have introduced ICT
based learning programmes have recognized these benefits and have attempted to provide
their employees with up-to-date technical knowledge and skills. All those interviewed
said that their employees need higher level and more specific skills for undertaking their
jobs (Scheuermann & Reich, 2002).

102
Characteristics of implementation
The following statements provide an overview of the main results of the case study in the
use of e-learning in Austrian SMEs. The statements point to the issues and problems for
SMEs in introducing e-learning.
• Learning needs were mostly identified through practical experience. Only a few
companies use programmes and questionnaires to identify skill and information
deficiencies.
• Learning is based in multiple locations and contexts including ‘on the job’, on the shop
floor, away from the job and in training centres depending on needs and possibilities.
E-learning takes place predominantly at the work-place and in specially equipped
rooms in the enterprise. Learning takes place during and after working time.
• Companies do not always have a choice in the selection of ICT-based learning
materials and programmes. Many SMEs depend on larger enterprises to provide
learning materials, e.g. simulations and technical information. Out of the SMEs that
are able to select their learning materials most do not analyse the market before
selecting the programme but ‘spontaneously’ choose a product that seems to fit to
their needs. In most cases they do not develop e-learning materials themselves. These
are usually provided by suppliers or external manufacturers.
• Enterprises have very different approaches in selecting their students for training
programmes. Often this is based on the skills and interest employees have in e-
learning. In other cases groups of employees (from a particular office or workgroup)
are selected to participate in an e-learning programme. In a third approach new
employees or employees taking on new jobs are enrolled on an ICT based learning
course The background of the learners is therefore very diverse, as is their prior
experience of learning.
• The main pedagogy deployed in learning in the enterprises is ‘learning by doing’
combined with face-to-face meetings. Employees are expected to become
autodidactic learners through e-learning.
• When trainers are involved in ICT supported learning they play the roles of tutors and
coaches. Typically, no additional qualifications are required for this role. The skills of
the trainers are based on their experience in traditional training.
• SMEs rarely evaluate their ICT based learning provision. If they do, they use one of
three methods (in order of usage): the evaluation of learning through practical
application, evaluation by the department in charge of e-learning or evaluation by an
external evaluator.
In general, those companies providing e-learning are satisfied with the outcomes but most
have no plans to expand e-learning provision. Out of the enterprises that make no use of
e-learning, only a few plan to introduce ICT based learning in the future.

Problems in introducing e-learning in SMEs

Enterprises have to rethink their business and learning practices in order to facilitate the
implementation of e-learning. The introduction of e-learning is often accompanied (or
triggered) by problems in the management of the enterprise, in the learning culture within
the SME or in infrastructure of the enterprise – to name but a few. These problems have
to be addressed in advance, otherwise e-learning will not be effective at an enterprise
level.

103
Management problems
Shortage of human resources is a major problem for Austrian SMEs. Often decision-
makers think - independent of size and sector - that their company does not fulfil the
prerequisites for e-learning, e.g. people do not have sufficient time or the learning
environment is unsatisfactory.
Decisions to introduce e-learning often do not depend on objective decision criteria but
are related to the attitude of the person(s) responsible for training. If ICT-based learning
is introduced the lack of knowledge and skills often results in a narrow concept of the
benefits and use of ICT based learning, with, for example, too strong a focus costs
Owner managers of SMEs have their own learning characteristics and are often reluctant
to devote time to formal learning.
Decision-makers in enterprises have to develop new models and methods for learning.
They have to learn to benefit from new emerging technologies and applications that are
playing a more and more important part in education (learning software, network-
technology, e-mail, application-sharing, etc.). However, many small and medium sized
businesses only have limited financial and time resources allocated to staff training and
the prospect of loosing staff for one or more days a week is not a popular option. The use
of ICT based learning could be a solution rather than the problem it is presently viewed
as. It allows staff to study at the workplace (therefore avoiding travel time and costs), use
specific learning materials tailored to user needs and to acquire know-how when needed.
It is understandable that many employers are reluctant to provide training that may
provide individuals with accreditation and the opportunity to move to higher positions in
other companies, but many manager owners of SMEs fail to recognise the added value of
the further qualification of employees to their company (Scheuermann & Reich, 2002).
One problem encountered in Austrian SMEs is described in detail by Goolnik (2002). He
points to the importance of the business experience of the sector in which employers
operate. Leading figures and organisations in a particular sector have more impact than
those outside it (e.g. universities, colleges and other training advisers and providers).
Their example can be either positive or negative. These leading persons and organisations
have to be a primary target group if the issue of e-learning for SMEs in a particular
region are to be addressed. The Austrian study showed no support for e-learning from
such influential actors.
One major step forward would be to persuade key individuals in SMEs of the potentials
of e-learning. Many personnel managers and training directors said they would like to
know more about examples of good practice. The study showed the need for more
information and training courses on e-learning (Scheuermann & Reich, 2002).

Understanding the potential of e­learning 
The learning culture in enterprises is a major influence on the quality of ICT based
learning. Hierarchies, power balances, controls, or the general esteem of training in SMEs
can be major obstacles to the successful application of ICT based learning. Enterprises
have to develop a learning culture where e-learning does not mean the same as playing
games on the computer but is honoured as self development and as a means to
strengthening the company. The learning culture also embraces the longer term planning
of learning activities and the embedding of ICT based learning in broader training
concepts. In contrast to this concept, employees are often forced to update their
competencies by themselves. This is a very difficult task for many. As a consequence,
Austrian enterprises often try dismiss those employees who are not able to adapt their
know-how to deal with new emerging trends and technologies. Learning culture also

104
implies an understanding of qualification that is connected to real needs in working life.
Certificates and formal qualifications only form a part of qualification. It is becoming
ever more important to acquire skills and knowledge in a short time for special purposes
(Scheuermann & Reich, 2002). ICT based learning directly addresses the continuous
improvement of individual qualification, which is difficult to assess externally. Decision
makers and managers lack insight into the learning needs of their employees and have to
change their understanding of qualification. Employees that are engaged and interested in
learning often feel that they do not get enough support and encouragement from their
superiors. ICT based learning can be made more attractive through financial and other
forms of recognition. Qualification, learning and further training should be recognized as
essential components of culture and development in enterprises (Hipwell, 2000).
Such a new orientation requires more strategic thinking. At the moment many problems
are caused by the informal and unsystematic approach of SMEs to ICT based learning
which often results in poor implementation and use of learning opportunities. Research
by the DELOS project (Developing Learning Organisation models in SME clusters) into
organisational learning in SMEs shows that most SMEs can be described as 'crisis-driven'
with little evidence of organisational learning. Information gathering practices,
knowledge acquisition strategies and competence development appear to be either absent
or rudimentary, and the enterprise typically responds to challenges and opportunities
rather than pursues an active policy of human resource development and strategic
management. The evidence suggests that this type of firm constitutes the largest category
of SMEs (http://www.pjb.co.uk/npl/bp19.pdf).

Inadequate infrastructures
Lack of organisational and spatial prerequisites
Austrian SMEs often lack the organisational and spatial prerequisites for ICT based
learning. For small enterprises it is often too expensive to adapt rooms or buy equipment
for e-learning on-the-job. Employees working in industrial and technical jobs lack
suitable workplaces and have to contend with unsuitable conditions for e-learning.
Therefore it is necessary to develop new environments for learning, e.g. the construction
of learning corners, learning islands or learning centres. It is important that employees
have immediate access to information in order to obtain the qualifications needed for
their work. Hence it is also necessary to analyse what help SMEs need to establish
environments and structures that support ICT-based learning.
Some spatial and organisational limitations could be avoided by combining e-learning at
home with work (see Kräutler, 1999) and to allow open, flexible access. In the 2nd
quarter of 2002, 62% of all Austrians over the age of 14 had access to a PC at home and
45% had internet access (http://mediaresearch.orf.at/internet.htm). Although these
statistics suggest that some enterprises could try to outsource learning to their employees,
none of the enterprises included in the survey had considered the potential of home based
e-learning.

Learning materials
Although the market for ICT based training is growing fast, there is a lack of adequate
learning materials for small enterprises. This is due to the following reasons. SMEs are
often unable to articulate and scope their learning needs. There are difficulties in
assessing the merit and value of available programmes and learning materials, which are
often perceived as failing to meet firm-specific needs. Finding appropriate training is also

105
made more difficult by a culture clash with external training providers, especially in the
public sector, who are seen as unable to understand business processes. Micro SMEs
have particularly problems with the cost of training programmes (and associated travel
and subsistence) and problems in releasing staff (Pye et al., 2002).
SMEs lack the time and financial resources to undertake in depth research into training
programmes that fit to their needs and the market for learning materials and course
providers lacks transparency. As an example, in Germany 35,000 training centres,
Chambers, and labour market organisations offer approximately 400,000 courses and
seminars (Siebold, 2002)!
Many firms are highly specialised. That makes the development of learning materials
expensive and commercially unattractive. One company included in the study said that
although learning materials were provided by a supplier they failed to meet the needs of
the company in a geographical and economic context (Scheuermann & Reich, 2002). If
enterprises find suitable learning materials they are didactically and methodically of poor
quality in many cases. Most criticism was of the complexity and lack of user friendliness
of products and also of the costs and time required. Quality certification of products
would probably foster development of better learning materials.
The availability and use of information and communications technologies and associated
services to facilitate the entire range of communication, interaction and transaction
(‘connectedness’) is poorly developed. E-learning is still seen as state-of-the-art
technologies.
From a pedagogical point of view e-learning can be problematic as it ‘facilitates’ isolated
learning. Learning in an enterprise is a cooperative process taking place through the work
process of the learners. New learning media have to integrate tools that support a more
cooperate approach: through the learning media it should be possible to communicate
with experts in and outside the enterprise, with tutors and coaches and other learners.
For a real integration of learning in work processes it is necessary to learn with real data
and real projects emerging from working life itself instead of examples and constructed
case studies. Learning systems have to be implemented within the work flow connected
to the enterprise databases (D’Atri & Pauselli, 2000).

Equipment
Whilst small enterprise in particular are concerned that they lack modern technologies for
e-learning, in many cases technology is less the problem than the human resources (users
and trainers). They lack the knowledge an skills to apply ICT based learning.
The use of technology to enhance or broaden the learning experience has until now been
looked at in too narrow a way. What is needed are ‘good models of learning' which are
enhanced by technology. Whilst reviews of research show that there are excellent pilots,
they lack critical mass, secure funding and therefore the likelihood of transferability.
Nevertheless, there is still demand for the development of simple and cheap solutions that
facilitate learning on the job. Laptops, Tablet PCs, handhelds etc. encourage flexible,
mobile learning but often lack linkage to other information and knowledge resources.
Although wireless networks can already be found in several Austrian SMEs, most
enterprises still have fixed wire networks and lack the flexible infrastructure for learning
on the job (Scheuermann & Reich, 2002).

106
Missing organisational perspective
Cooperation of different actors
At present most Austrian SMEs act alone in facing problems of learning. For future
development, it is necessary to strengthen cooperation between SMEs and between
SMEs, larger enterprises and public institutions (e.g. Chambers of Commerce).
Development in this field has to aim at two goals:
•SMEs cannot afford the infrastructure to provide ICT based learning
environments or the production of tailored learning materials. If SMEs
pool their resources they can afford high quality products. Cooperation in
that sense could help SMEs to reduce costs and share know-how.
•SMEs often need specific know-how or information that could easily be
obtained through information and communication technologies. In many
cases there is even no need for structured learning materials if video-
conferencing or other communication tools are available. Cooperation
could lead to the sharing of information and know-how and lead to
collaborative forms of learning.
Austrian SMEs cooperate with other enterprises in e-learning if they are suppliers. Some
SME managers said they had profited greatly from this cooperation. On the other hand,
learning materials provided by larger enterprises are often standardized and do not take
into account local needs. Furthermore, the supplier tries to pack as much information as
possible into the learning materials making it difficult for a small enterprise to filter
necessary information (Scheuermann & Reich, 2002).
One solution may be to develop clusters or networks of SMEs. Co-operation between
organisations within markets has long been identified as a factor in economic success and
networking between organisations can contribute to stability and reduce uncertainty.
These networks can evolve over time as ‘natural’ clusterings of enterprises, or can be
‘induced’ artificially as a result of interventions such as the development of business or
science parks.
As a large proportion of SMEs are in ‘crisis management’ rather than pro-active learning
situations, they need to be encouraged to adopt a more participative style of collective
learning. Support services need to be provided and resources shared (Hawke, 2000). No
evidence could be found that Austrian SMEs build up such co-operation or are engaged
in regional learning networks as described by Stahl (2002).
He suggests the learning region can promote local change, empower SMEs through
networks and partnerships and foster innovation. This concept could be extended to
learning, especially considering that SMEs are consumers of education and training with
particular requirements and constraints.
Partnerships and networks at local and regional levels could certainly stimulate new
experiments, actions and directions for learning. Clusters of SMEs, larger enterprises and
public institutions that already exist in most European countries could act as starting
points for the construction of learning networks. The government in Austria and the
regional government in Tyrol have launced initiatives aimed at creating regional clusters.
These clusters are built up by a group of enterprises in a specific branch or sector; e.g.
Alpine technologies clusters in Tyrol (http://www.alpine-technologies.com). Informal
knowledge transfer is already taking place in such networks but could be expanded into
other areas.

107
Training provider 
SMEs are by nature of their structure and competence often unable to provide training
themselves. In many cases training provided by an external education enterprise is a
more effective solution for SMEs than internal training (Scheuermann & Reich, 2002).
However the provision of training and consultancy has to take into account the specific
characteristics of SMEs. These include the involvement of the management at the
technical and operative levels, close personal relationships between managers and
employees patriarchal style of running the enterprise, a low degree of formality and
organisation and the lack of hierarchies. This leads to specific problems, including a
‘dominance of daily business’ (and thereby neglecting co-operation with the consultant)
and a lack of information about the economic and treading environment. The demand for
consultancy is often reactive and depends strongly on the attitude of the SME owners
and managers towards consultants (Kailer, 2000).
As Stahl points out, modern enterprises require learning as an integrated activity. That
means that external training institutions have to act in a customer-oriented way in order to
foster these activities (Stahl, 2002). As we have discussed above, there is still
considerable confusion in SMEs over the role of learning and human resource
development in management policy. This requires capacity and competence in providing
consultancy. Because of material restrictions there are also limitations in relation to
human resource development. Most of them are financial and organisational (it is never
easy for enterprises to send their staff to external, long-term seminars). Training
institutions have to deal with these problems creatively. They should view these
restrictions as challenges to foster innovative solutions. Training institutions need new
concepts to meet these challenges. Close co-operation with SMEs is needed to deal with
these problems and to integrate the activities of the external institution into the enterprise.

Conclusion
The present EU definition of an SME is too general to be of great use in research terms.
It fails to recognise the qualitative criteria that can be seen as typical for SMEs, e.g. the
majority of SMEs are family enterprises. Small and medium sized enterprises embrace
very different forms of organisational structures and work in very different branches and
sectors – from a traditional handicraft enterprise with less than ten employees to an
innovative enterprise with 250 employees and several branches. It is obvious that these
different kinds of enterprises have different demands for knowledge and knowledge
management (OECD, 2000). If better e-learning solutions are to be found, groups of
SMEs have to be identified based on quantitative and qualitative criteria. This is
necessary as it is too expensive to develop solutions for individual SMEs.
SMEs have to develop new organisational structures for the implementation of
cooperative and collaborative forms of learning. At the moment most SMEs implement
isolated learning solutions that fail to meet the needs of cooperative and collaborative
forms of working and learning. A further step in e-learning would require investment in
technology that supports synchronous as well as asynchronous communication at the
workplace. This is probably too expensive for most SMEs but costs could be reduced by
cooperation in investment in technology and the sharing of information.
New approaches are needed in terms of understanding the broad context of learning.
Pedagogical approaches in Austrian SMEs are often inadequate and do not meet the
needs of e-learning. There is also a lack of flexible learning solutions in SMEs.
Furthermore, there is still much to be discovered about how people learn using different

108
technologies, particularly in relation to interactivity, and how materials can be developed
and structured to enable all learners to make effective use of them.
Learning at the workplace should partly replace “old” teaching in classrooms and face-to-
face courses away from the enterprise, but this is difficult to implement. Learning should
not take place besides work but with and through work. Learning at the workplace is
different from learning at school and in a classroom. It is not important to go through a
subject systematically but instead to solve problems arising from practice. Therefore
flexible learning media are needed.
The term “e-learning” itself is somewhat ambiguous in a world that is rapidly changing.
Where does e-learning start and where does it end? Where is the divide between e-
learning and informal learning and is it necessary to differentiate between them?
As technologies develop there will be far greater access to learning opportunities.
However, our study has shown that considerable work is needed in order to ensure that
the new e-learning opportunities are developed and implemented within SMEs.

References
D’Atri, A., Pauselli, E.: Distance learning for SME Managers. 2000. Retrieved:
19.12.2002. http://cersi.luiss.it/articoli/ITHET2000.pdf
DELOS (Developing Learning Organisation models in SME clusters): Final report.
http://www.pjb.co.uk/npl/bp19.pdf. Retrieved: 16.12.2002.
Goolnik, G.: E-Learning for Smaller Rurally Based Businesses: A Demand-Led
Challenge for Scottish Educational Institutions. Online Journal of Distance Learning
Administration, Volume V, NumberII, Summer 2002. Retrieved 14.12.2002.
http://www.westga.edu/~distance/ojdla/summer52/goolnik52.html
Hawke, G.: Factors Influencing Active Learning in Small Enterprises. Paper presented at
the Adult Education Research Conference (AERC) 2000. Retrieved 14.12.2002.
http://www.edst.educ.ubc.ca/aerc/2000/hawke1-final.PDF
Hipwell, W.: Promoting your e-learning investment. 2000. Retrieved: 19.12.2002.
http://www.learningcircuits.org/sep2000/hipwell.html
Kailer, N.: Co-operation between SME and consultants: Analysis of deficits and starting-
points for improvements. 2000. Retrieved: 18.12.2002.
http://www.ifgh.ac.at/kmuforum/20001/20001_06.asp
Kräutler, W.: Information-Society-Technologies: Chancen für Tirol. In: Tiroler
Zukunftsstiftung (ed.): Nicht ohne Netz. Tiroler Unternehmen auf dem Weg ins Internet.
Innsbruck, 1999.
Oberholzner, T., Sheikh, S.: Innovative Small and Medium Sized Enterprises and the
Creation of Employment. Vienna, 2001. Retrieved: 12.12.2002.
http://www.ifgh.ac.at/de/lot9.htm
Organisation for Economic Cooperation and Development (OECD): Small and Medium-
sized Enterprises: Local Strength, Global Reach. 2000. Retrieved: 28.11.2002.
www.oecd.org/pdf/M00005000/M00005918.pdf
Pye, Jo et al.: Promoting Workplace Learning with ICT: Modes and Models for
Organisational Change. Edinburgh, 2000. Retrieved: 14.12.2002.
www.leeds.ac.uk/educol/documents/00001611.htm
Scheuermann, F., Reich, K.: E-learning in Austrian SMEs. Innsbruck, 2002. In press.

109
Siebold, Heinz: Der vierten Säule des Bildungswesens gehört die Zukunft. In: Das
Parlament. Nr.: 31-32. 2002. Retrieved 28.11.2002.
http://www.das-parlament.de/2002/31_32/Thema/039.html
Stahl, Thomas: The learning region and its potential roles in lifelong learning. Retrieved:
14.12.2002. http://www2.trainingvillage.gr/download/publication/panorama/lllEN.pdf
Zugmann, J.: Die Zukunft virtueller Klassenzimmer. In: Der Standard, 13.10.2001.
Retrieved: 13.12.2002. http://derstandard.at

110
Assessing the application of online learning in work-
based setting

John Munro and Philip Crompton, University of Stirling (UK)

Introduction
The UK Government has championed the concept of lifelong learning for organisations
and their employees as a key strategy for developing a competitive economy to meet the
demands of the global market. The importance attached to learning, by the UK
Government, as a strategy to develop competitive businesses has been mapped out in its
Green Paper 'The Learning Age' (DfEE, 1998). The Green Paper promotes the view that
a) learning assists organisational development by increasing the skills base of the
workforce, b) through the process of learning itself, it enables the organisation to manage
and respond to change, and c) it increases the knowledge base of the organisation, which
is seen as the key to discovery and innovation. As a consequence, developments in work-
based learning are seen as particularly appropriate to increasing the development of
workplace skills and knowledge, whilst at the same time widening access to learning
(Seagraves et al, 1996).
In tandem with promoting lifelong learning, the UK Government emphasises the role of
Information Communication Technologies (ICTs) in contributing to competitive gains by
small and medium sized firms (Southern and Tilles, 2000). According to The Scottish
Office policy document ‘Opportunity Scotland: A Paper on Lifelong Learning’ (1998)
the exploitation of ICT will be key to the activities of the recently established Scottish
University for Industry (SUfI) in seeking to 'boost competitiveness and individual
employability'. However, compared to the United States of America, the introduction of
online learning using ICTs is present only within a small minority of United Kingdom
and European companies (Pye, 2000).
Seagraves et al (1996) relate how the daily operational needs and pressures within SMEs
can adversely affect the opportunities and time that is required for effective learning to be
developed at work. Other commentators have similarly reported how problematic
structured learning in SMEs is due to the constant workload pressures that exist (Ram,
2000; Bridge, O'Neil and Comrie, 1998; Gibb, 1993). As Lange, Ottens and Taylor
(2000) argue, it is not the issue of learning itself that is problematic for SMEs, but rather
the 'luxury of allowing staff already at stretched capacity to take part in formal training'.
Therefore, much of the learning that is done in SMEs is more likely to be unplanned,
informal, and tailored more to matching and overcoming the daily operational demands
and crises (Westhead and Storey, 1999; Gibb, 1999; Gray, 1999).
At the same time, studies by Seagraves and Osborne (1995), Seagraves et al (1996) and
Loots, Osborne and Seagraves (1998) highlight how the assessment of learner needs must
address the issue of learner motivation and commitment to learning in the work-based
setting. Indeed, as extensive studies by Boshier have shown, there is a link between
learner motivations to study and the attendant drop out rates (Loots, Osborne and
Seagraves, 1998).

111
Information Technology
The proliferation of the use of the worldwide web as an educational medium has seen the
adaptation of the computer as a means of communication between learners. The web has
matured from merely a provider of pages of information to an interactive environment
through the provision of email, chat rooms, and web-boards. Many courses are now being
developed directly onto the web and range from simple course notes to complete online
courses in virtual classrooms. Online tutorials are not just limited to distance education
courses but are also finding a place in regular university courses. The use of computer-
driven interaction through email etc. is one of the fastest growing forms of
communication (Palloff and Pratt 1999).
According to recent research computer conferencing is said to offer many benefits that
aid student learning and improved social relationships (Crook, 1994; Mercer and
Wegerif, 1999; Ryan et al, 2000). The use of computer conferencing allows learners to
present issues, clarify misunderstandings, prepare learners for new topics, engage in
ongoing debate and discussion and deliver time-consuming administrative information
(Mowker, 1996). This approach has lead to the increased adoption of co-operative and
collaborative learning through the use of new technology. Instead of merely asking
learners to write individual essays it is now possible to allow students to produce
presentations including graphics, audio, video and other forms of multimedia; resources;
data files etc. However, the amount of work required is too much to expect from an
individual but the technology allows the students to work co-operatively in order to
produce such work.
Co-operative learning refers to learners working in small groups to achieve a mutually
understood task through a process of interaction. Underwood and Underwood (1999) see
co-operative learning as a mechanism that
“emphasises cognitive processes such as conflict resolution, hypothesis
testing, cognitive scaffolding, reciprocal peer tutoring and overt execution
of cognitive and meat-cognitive processes and modelling.”(p. 12)
McKendree et al. (1998) refer to the use of students acting as a resource for each other
and that their online conversations can be captured for use by future learners. Mayes
terms this use of preserving digital conversations for others to read as ‘vicarious learning’
(Mayes, 1997). The efficiency of these group interactions can be due to the use of inbuilt
conflict set into the activity in order stimulate the learners to engage with each other as
detailed by Piaget. The alternative mechanism is through the use of Vygotskyian co-
constructive processes.
Underwood and Underwood (1999) suggested that interaction between learners in the
form of computer conferencing might be more effective in achieving learning outcomes.
There is evidence to suggest that peer tutoring and peer collaboration can effect learner
development. If a group is effective then there will be an exchange of knowledge and
learners will accept information from each other as well as requesting information and
assistance. There are very few studies on the effectiveness of computer-mediated
conferencing (CMC) and detailed analyses of the nature of learners’ postings in an online
instructional work-based setting.
The hope that e-learning might revolutionise learning in the workplace has, so far, not
been realised. Oberski et al (2000) also concluded that the continued workload pressures
and the juggling of priorities made e-learner participation difficult to maintain. At the
same, time lack of familiarity with the use of computers and the Internet are issues for
many would-be learners. Information Technology brings its own array of technological
issues and barriers that need to be faced by online learners (Berge, 1998). For example,
problems with connectivity, slow Internet response rates, inadequate software and

112
hardware for the downloading of documents, sound and video files, and inadequate or
inaccessible technical support.
A review of the literature on small and medium sized business learning shows that
research data is sparse and limited (Chaston, Badger and Sadler-Smith, 1999). This is
also reflected in the limited, but growing research on the application of Information
Communication Technologies (ICTs) in the workplace and within SMEs in particular
(Southern and Tilles, 2000; Gray, 1999; Teague, 1999). This particular research project
was designed to provide a case study of a work-based learning experience, using ICT
Web-based online learning access, within a medium sized enterprise in the engineering
sector of the Scottish economy. The company’s manufacturing output includes:
machining, presswork, sheet metalwork, metal finishing and electro-mechanical
assembly; and supplies most of its products to major computer hardware manufacturers
around the world. The company holds the Investors In People (IIP) award and believes in
maintaining a competitive advantage through a policy of training and developing a highly
skilled and flexible workforce. In this study, 51 employees out of a total of 226 staff
participated on an online learning module on finance that was part of the post-graduate
Certificate in Small and Medium Enterprise Management (CSMEM), provided by the
University of Stirling.

The company’s aims

♣ For the employees to develop awareness and understanding of the
company's financial performance.

♣ Promote development of basic Internet/Web skills by the

employees.

♣ Encourage a culture of lifelong learning within the workforce.

♣ Encourage employees to enhance their personal financial

management skills.

The objectives of the investigation

The objectives for this particular investigation were:

♣ To investigate if employees within the company could learn to develop a quality

questioning approach, as an indicator of learning, in order to understand and
interpret the financial information presented to them by the company, through
participating in an online learning course on finance.

♣ To evaluate the work-based learning experience of staff using a web-based e-

learning experience.

♣ To gain further knowledge and understanding of the issues that surrounds the
process of learning within a medium sized enterprise.

♣ The case study ran from October 2000 to February 2001 inclusive.

113
Methodology
A qualitative case study was designed and implemented over two phases using a before
and after approach. Participants received their initial questionnaire, Questionnaire 1 prior
to starting the CSMEM online Finance Module. Throughout the duration of the course
the number of recorded ‘hits’ by participants helped to identify the level of participation.
This demonstrated that most employees were not active over the period of the study. The
outcome being that 79% of the sample never logged-on and only 29% actively
participated. Two separate focus groups were issued with the second questionnaire,
Questionnaire 2, and these groups took part in semi-structured interviews at the end of
the course. The Active Focus Group was representative of those who had had made some
progress in the course, whilst the Non-Active Focus Group consisted of those who had
not actively participated. The purpose of these Focus Groups was to draw out issues from
both groups, which highlighted the experience of those actively participating, and issues
that arose out of lack of participation. It was the intention that this investigation would
identify any links existing between learning finance and the quality of questions that
employees would subsequently ask their employers. The final stage was to hold a semi-
structured interview with the Human Resource Manager to outline the views and
perspective of the management following the particular learning experience.

Pre-course investigation
A representative sample of 51 people received Questionnaire 1. This was the total
number of participating employees who were registered on the WebCT CSMEM Finance
Module. At the time of sampling, the company’s total workforce was comprised of 226
employees of which the sample size was 23%. The overall response rate was 53%
resulting from the final sample size of 27 returned questionnaires. The questionnaire
indicated that all the respondents recognised the value of computers and how the Internet
could help with their learning. Of these, 96% had prior experience of using a computer
and were at ease with this mode of learning, in fact 63% of the sample had their own
personal computers at home.
Thirty percent of the sample anticipated that undertaking a course on finance would help
them to understand the company’s financial performance. The largest response of the
sample (63%) felt that they needed to understand the company’s financial performance in
order to determine the company’s profitability, and perceived their own job security as
being linked to this.
After the questionnaire had been returned a semi-structured interview was conducted
with a Focus Group to explore employee pre-course thinking. At the time of sampling
and interview, the management perceived that employee questions on finance were
always related to aspirations for improved pay rates and more opportunities for overtime.
It was the management’s hope that a course of finance might bring some revelation to the
thinking of the employees, and hopefully the company’s figures would produce more
awareness as to why employee’s aspirations could not be fulfilled. The results from the
questionnaires did not support the management’s view of employee thinking, nor did the
Focus Group interview. Rather it was the view of the employees that understanding
finance would help them determine if the company was profitable, and if so, then their
jobs would be secure. At the time of the study, Autumn 2000, the company was starting
to experience a downturn in demand for its products from major manufacturers in the
USA. It is likely that at the time of sampling this attributed to the workforce’s pre-
occupation with job security and how they were seeing it impact on the production line.

114
Post-course investigation
Thirty-eight people remained registered as participants for the WebCT CSMEM Finance
Module. Questionnaire 2 was distributed to this remaining group who were representative
of 18% of the company’s workforce, which had reduced to 210 employees. The reduction
in the total workforce resulted from a redundancy programme ushered in by the downturn
in the information technology sector reducing demand for the metal casings made by the
company, and partly due to some participants leaving to take up new employment. The
final sample size of 25 completed questionnaires provided a response rate of 65% from
the participants.
The relevance of the subject (finance) was a primary concern for the majority of the
participants, motivating only 20% of the sample to participate in the course of learning.
The fact that is was not viewed as job-related learning explained the low participation. It
was not seen as a means of helping them to perform better in their jobs, nor was it likely
to help them manage their personal finances. If they had been given a greater choice of
job-related subjects they would have been more motivated in engaging learning at work.
Given adequate time, encouragement to learn, and the right subject, work-based learning
was considered as a potentially worthwhile experience. However, in this particular
leaning exercise the employees considered learning about finance to be something the
company wanted them to do. They viewed it more as an imposition rather than an
opportunity for learning.
Furthermore, the second questionnaire and the follow-up Active and Non-Active Focus
Group interviews clearly indicate that the majority of employees do not accept it is their
responsibility to learn how to understand the figures. They expect the company to provide
clear and unambiguous figures, and also to present them in an easily understandable
manner. Very few of the employees used the figures as a bargaining tool for increased
pay, overtime and more bonuses, which was actually contrary to the expectations of the
management. As with the pre-course findings, the employees anticipated that a greater
understanding of the company’s figures would have given them evidence of company
profitability relating to their own job security.
The small number of employees actively participating in the course on finance meant that
it was impossible to determine if a link existed between the type of questions asked
before and after the course of finance. Indeed as the Active Focus Group members
reported, their own perceptions of the company's figures had not altered in any way as a
consequence of participating on the Finance Module.
Participants appreciated the relevance of Information Technology and were more
interested in how to use computers in the workplace, develop programming language
skills, and production machinery programming. Central to their thinking was the need to
keep abreast of technology developments for improving their productivity and for
ensuring the company maintained a competitive edge in the marketplace. This very much
supported the management’s view of the importance of training and development in
ensuring future business success. Previous studies by Seagraves et al (1996) and
MacLaren and Marshall (1998) have shown that the motivation to study by work-based
learners is more often driven by learning that is applicable to their work and how it
enables them to do their jobs more effectively.
e-learning was stimulating and increased interest in learning technologies and learning by
the Internet. Forty eight percent of the sample considered this mode of learning to be
appropriate and staff would be able to keep abreast of work-related technology
developments by having access to learning technologies. Clearly, an interested in

115
learning has been created by the implementation of online learning. Indeed, one technical
supervisor enthusiastically indicated it brought the learning to him by providing an
environment and opportunity to learn without having to travel off-site which was
difficult, as he did not have a driving licence and public transport was poor. In this
employee’s case, transport was a real barrier to pursuing learning. Therefore the
availability of online learning made it possible for him to access opportunities for
learning that were previously impossible. The Human Resources Manager considered this
to be a good example of how an interest in learning can be fostered given the right
support and access to learning.
One drawback of the learning was the slowness of the Internet. When participants had to
wait sometimes for online questionnaires to be completed or for documents to be printed
off they felt de-motivated. Although e-learning was useful, many preferred peer learning
support, rather than the isolation of learning on their own in front of a PC. Another source
of de-motivation was the isolation involved. The company had provided a small room
with two PCs for participants to book time on for their learning to take place. It was their
view that had they been given the opportunity to study together as a group in a computer
classroom they would have been able to share problems and together identify suitable
solutions. For example, when they experienced difficulties getting to grips with some of
the financial exercises or when they encountered problems with the online technology,
had they been able to engage in problem sharing and problem solving with colleagues
face-to-face then this might have reduced personal frustrations with the learning
experience and alleviated levels of de-motivation. This may indicate a need for a
workplace champion to actively promote e-learning within the organisation, and who will
encourage staff to pursue a course of learning, by providing support and helping them to
overcome difficulties.
The course would have been better designed had it made use of a central online
discussion area that would have enabled the learners to leave messages for each other
thereby enabling them to communicate with each other. Thereby allowing them to share
their problems and solutions. As it was shared communication was less structured and
patchy through conversations in the corridor or at the coffee machine. Failure to make
use of this technical feature in the course was down to poor course design rather than a
technological deficiency.
It was also not surprising, as Seagraves et al (1996) and Oberski et al (2000) found in
previous studies, that time and workload pressures made the greatest impact on
participation and motivation. Sixty percent of the sample attributed time and workload
pressures as the main reason for either not participating in the course or for restricted
participation. This was further confirmed in both focus group interviews. Most staff
taking dedicated time off to learn were hindered by the culture of the workplace, which
tended to act against their desire to participate in the course. Pressures of work and daily
operational demands took priority over learning despite the company offering an hour off
work each week (to be matched by the employees giving an hour of their own time).
There is evidence from the questionnaire responses to suggest that there is a genuine
interest in lifelong learning given the right conditions and the right subjects to foster it.
Yet in order for this to be encouraged in the workplace, operational demands need to be
addressed before there can be any hope of the employees embarking on effective work-
based learning.
With reflection, it is was agreed by both the employees and the Human Resources
Manager that it might have been more beneficial to have staggered the course rather than
having so many people starting at the same time. The learning could have been offered to
a smaller pilot group, which would have given an opportunity for any problems to be

116
sorted early on in the programme, providing a more positive experience for those joining
later on. The size of the initial group (51) made it difficult to provide effective support
within a busy operating environment. Clearly, more thought needs to go into how and
when people learn. Although the company had requested that employees would complete
the course in six weeks during the period October to December 2000, this proved to be
unrealistic and so the period of learning required extension to February 2001. The
management had not anticipated the effect of the approaching Christmas season as well
as the operational demands on the employees. It proved a busy and stressful time,
resulting in much reluctance to give an hour of their own time after work to pursue their
studies.

Summary
From the outset, the results generated from the case study were not considered to
representative of all SME experiences of online learning in a work-based setting.
Nevertheless, this study did highlight the experiences specific to a company and its
employees as a result of implementing e-learning within a work-based setting with the
hope that the findings will enhance knowledge in the areas of both work-based and online
learning. It is hoped that if all employees eventually completed the course, a longitudinal
study would reveal if there was any link between completing the course and the nature of
the questions asked at the end of the course.
As mentioned earlier, work-based learning and online learning are considered to be
important ways of boosting learning for the individual and enhancing company
performance. However, this study reflects the common problems that can arise for small
to medium sized enterprises particularly in the areas of time and workload pressures.
The investigation has shown that when designing courses for learning it is important that
they have relevancy in helping people to do their jobs more effectively. Furthermore
greater consideration needs to be given as to how courses are converted for online
delivery. What had been considered a satisfactory course when delivered face-to-face had
problems when merely converted to a web format without due consideration being given
to the design and delivery of the course from an e-learning point of view. Allowing
students access to their fellow students both to discuss course matters and also for social
exchanges we consider a crucial facet of learning online. This feature allows for the
promotion of a Community of Learning that acts as a support network for its members.
It was also seen from the study that the working environment is an important factor in
influencing the motivation and interests of the employees. Even though it is laudable to
encourage learning, the issue of relevance cannot be ignored. However, on the positive
side Web-based learning was seen to be a good way of accessing learning opportunities
related to how well they could perform in their jobs.

References
Berge, Z.L. (1998) Barriers to Online Teaching in Post-Secondary Institutions: Can
Policy Changes Fix It? Online Journal of Distance Learning Administration Vol 1 (2) pp
1-12. Found at: http://www.westga.edu/~distance/Berge12.html [Accessed 4 April 2000]
Bridge, S., O’Neil, K., and Cromie, S. (1998) Understanding Enterprise,
Entrepreneurship and Small Business. Basingstoke: Macmillan Press.

117
Chaston, I., Badger, B., Sadler-Smith, E. (1999) Small Firm Organisational Learning:
Comparing the Perceptions of Need and Style Among UK Support Service Advisors and
Small Firm Managers Journal of European Industrial Training Vol 23 (1) pp 36-43.
Crook, C. (1994) Computers and the Collaborative Experience of Learning. London:
Routledge
DfEE (1998) The Learning Age. Sheffield: Department for Employment and Education.
Gibb, A. A. (1993) Small Firms Training and Competitiveness. Building Upon the Small
Business as a Learning Organisation International Journal of Small Business Vol 15 (3)
pp 13-29.
Gibb, A (1999) SME Policy, Academic Research and the Growth of Ignorance, Mythical
Concepts, Myths, Assumptions, Rituals and Confusions International Small Business
Journal Vol 18 (3) pp 13-35.
Gray, D (1999) Work-based Learning, Action Learning and the Virtual Paradigm. Paper
presented at the European Conference on Educational Research, Lahti, Finland 22-25
September 1999.
Found at http://www.leeds.ac.uk/educol/documents/00001260.htm
[Accessed 1 August 2000]
Lange, T., Ottens, M., Taylor, A. (2000) SMEs and Barriers to Skills Development: A
Scottish Perspective Journal of European Industrial Training Vol 24 (1) pp 5-11.
Loots, C., Osborne, M., and Seagraves, L. (1998) Learning at Work - Work-based Access
to Higher Education The Journal of Continuing Higher Education Vol 46 (1) pp 16-30.
MacLaren, P., Marshall, S. (1998) Who is the Learner? An Examination of the Learner
Perspectives in Work-based Learning Journal of Vocational Education and Training Vol
50 (3) pp 327-337.
Mayes, T., (in press), Groundhog Day, Centre for Learning and Teaching Innovation,
Glasgow: Glasgow Caledonian University.
McKendree, J.; Stenning, K.; Mayes, T.; Lee, J.; Cox, R.. Why Observing a Dialogue
May Benefit Learning. Journal of Computer Assisted Learning; v14 n2 p110-19 Jun
1998. 1998
Mercer, N. and Wegerif, R. (1999) Is ‘exploratory talk’ productive talk? ’ in Littleton, K.
and Light, P. (eds) Learning with Computers: Analysing productive interaction. London:
Routledge
Oberski, I., Palomar, A., Noya, C., Ruggiero, E., Herrera, F., Korhonen, K., Osborne, M.,
and Davies, P (2000) Evaluating Online Work-based Education for Managers in SMEs.
Some Initial Observations. Industry & Higher Education Vol 14 (3) pp 200-203
Palloff, R, and Pratt, K., (1999) Building Learning Communities in Cyberspace: effective
strategies for the online classroom. San Francisco: Jossey-Bass Pye, J (2000) Promoting
Workplace Learning with ICT: Modes and Models for Organisational Change. Paper
presented at the European Conference on Educational Research, Edinburgh, 20-23
September 2000. Author contact: Exeter University.
Found at: http://www.leeds.ac.uk/educol/documents/00001611.doc [Accessed 4 March
2001]
Ram, M. (2000) Investors in People in Small Firms: Case Study Evidence from the
Business Services Sector Personnel Review Vol 29 (1) pp 69-91.
Ryan, S, Scott, B., Freeman, H, and Patel, D. (2000) The Virtual University: The 
Internet and Resource­based Learning. London: Kogan­Page

118
Seagraves, L., Osborne, M., Neal, P., Dockerell, R., Hartshorn, C., and Boyd, A (1996)
Learning in Smaller Companies Final Report. Stirling: Educational Policy and
Development University of Stirling.
Seagraves, L., and Osborne, M. (1995) Learner Motivation in Part-time Higher
Education including Work-based Learning Paper Presented at Towards A Learning
Workforce Conference, University of Lancaster, 12th to 13th September 1995.
Southern, A., and Tilles, F (2000) Small firms and information and communication
technologies (ICTs): toward a typology of ICTs usage New Technology, Work and
Employment Vol 15 (2) pp 138-154.
Teague, J. S. (1999) Computer Mediated Communication in Distance Post-Graduate
Teacher Education: Students’ and Tutors’ Perceptions of Different Types of Computer
Mediated Communication. Paper Presented at The British Educational Research
Association Conference, University of Sussex, Brighton, 2-5 September 1999. Found at:
http://www.leeds.ac.uk/educol/documents/000001100.doc [Accessed 12 November
2000]
The Scottish Office (1998) Opportunity Scotland: A Paper on Lifelong Learning.
Edinburgh: Stationery Office.
Underwood and Underwood (1999) ‘Task effects on co-operative and collaborative
learning with computers.’ in Littleton, K. and Light, P. (eds) Learning with Computers:
Analysing productive interaction. London: Routledge
Westhead, P., and Storey, D. J. (1999) Training Provision and the Development of Small
and Medium Sized Enterprises: A Critical Review. Scottish Journal of Adult and
Continuing Education Lifelong Learning. Vol 5 (1) pp 35-41.

119
120
A holistic vision of the future of e-learning

Kees Schuur, ??? (Netherlands)

Introduction
During the last 15 years there has been considerable research and development into the
use of Information and Communication Technologies (ICT) for learning. e-learning
environments have become readily available during the last 5 years. If we believe the
claimed advantages of e-learning, for example cost savings, Just-In-Time learning, any
time, at the learners’ own pace, at any place, flexible learning styles, etc, the future looks
bright. Despite this, only a small percentage of learners, especially in industry, are using
e-learning environments.
This article explores several factors, which influence the use of e-learning environments
from the point of view of developments in different disciplines and in society, and
provides a vision of the future for ICT based learning. It outlines for development of e-
learning environments in the future. It is not intended to be an exhaustive list , but rather
to show the direction of development and to be a starting point for further discussion.
Examples11 of different systems and platforms are included to illustrate the discourse.
What kind of impact will new technologies have on society? Where are we heading?
How can we shape the development of technology? And what will the impact be on our
learning and the instruments we use for learning, such as e-learning?
One way to create a picture of the future is to identify present technologies and based on
this to describe possibilities for the near future. A second way is to look back at historical
developments and to extrapolate from these into the future. A third way is to combine
elements from different disciplines and view different combinations as pictures of the
future. This paper tries to combine these three ways

The increasing speed of change, development and the

future
Changes: in the past, present and future
Create the New World,
Because we don’t understand it. Xerox, 1971
Life is dynamic and continuously changing. We would like to predict the future, but we
know we never can be sure. There are too many uncertainties, too many variables and
actors, which influence the developments taking place. The only thing we can be sure of
is that the future will be different and dynamic.
The speed of change is accelerating. In the past centuries, a change could be incorporated
into society over a time-span of several generations. In this century, changes have a time-
span of one generation; now changes are taking place within a generation. Changes in the
past could be managed, now they only can be facilitated and barely incorporated, in the

11
Sometimes brand names are used. These are only meant as an example and are not meant as an evaluation of that
product / brand. It only describes one small element of the instrument / environment, certainly not enough for positive
or negative evaluation of that product / brand.

121
future there will barely be time enough to initiate them. It seems that change just takes
place.

Time-scale in societal development

In the historical development of society a logarithmic scale can be seen. The
developments reviewed here have different time-scales. Some are measured in thousand
of years; others in hundred of years and still others have a time-span of only decades.
Future time-spans will most likely be years, months, weeks, or even days. It is possible
that future developments will have no longer have a direct impact on lifestyles. Change
has become part of everyday life. We will be living in a second or third mode.
Around 5000 years ago, people began to record their knowledge in writing on stones,
parchment and later on paper. Before that period knowledge was preserved in the
(collective) mind of individuals and groups. Knowledge was passed on from parents to
children, from elderly to young. Security and the physical survival of the species were the
most important issues in the agricultural age. Knowledge was expended on securing food
and housing. With this security in place, people could devote some of their time to other
activities such as art and technology.
A new phase was entered when society had to address more complex issues. Urban
centres emerged. The development of cities led to commerce and trade and
manufacturing. Knowledge, now made explicit in writing, was centred on libraries like
Nineveh and Alexandria, where scholars could come together to explore and combine
explicit knowledge and create new knowledge. These centres of learning were the basis
of the establishment of the universities.
Structuring information and knowledge became necessary as activities like warfare and
construction of buildings such as castles and churches became complex. Knowledge was
distributed first by hand-copied manuscripts and later by printed books and documents.
Explicit knowledge became available to more people.
The industrial age brought mass, low cost production. Society required a trained, efficient
and skilled workforce. Education and training systems were developed with mass school
and classroom based education. It was the age of social control. But knowledge was
becoming still more complex, with increasingly rapid change. What was learned at school
was already outdated by the time it could be applied in the workplace. In the working
process, knowledge was changing continuously. People were expected to participate in
lifelong learning. Possibly more important was the un-learning process, where people had
to forget what they had learned.
The focus of education moved from the classroom to group based constructivist learning.
Knowledge was broken down to small units and the timeframe for learning (especially in
higher education) changed from courses, which lasted sometimes for years, towards day
long seminars or workshops of less than an hour. The use of computers allowed storage
and access to large quantities of information and explicit knowledge.
In the last decade, access to information has become readily accessible. While writing
this article, Altavista gave more than 500.000 hits for ‘e-learning’ and more than 600.000
for ‘e-learning’. The Google search engine already scans over one billion URLs. Around
600 million people have Internet connections of which 190 million are in Europe.
Because it is impossible to read all this information or to contact all those people, the
future approach to knowledge will be through networking, in communities of practice
and on learning systems and processes within groups.

122
Personal development
The philosopher Cornelis describes the societal developments outlined above in
individual terms. In the first part of someone’s life, a person lives in a protected
environment, where he or she learns basic skills and social behaviour. In the second part
of their life they live in a socially controlled environment reaching its highest level at the
age when people get a job and become part of social groups (parents, yuppies, dinkies,
etc). Cornelis sees the third phase of life as when one has experienced a lot, the kids have
left the house, their income is assured. That phase he describes as the phase of
communicative self-control.
A problem is that the driving forces to use e-learning environments often come from
highly educated people in the social control system and from more elderly people in the
communicative self-control. These individuals and groups expect others to comply with
their criteria for education and technology. What does all this mean for e-learning and the
use of e-learning in SMEs. At this stage we can advance a number of ‘Learning-
discussion points’.

Learning- / discussion points:

− e-learning in SMEs should offer possibilities for ordering and classifying the
most important information and knowledge and making it accessible both
technically and through mental schemas.
− In the age of networking, networks are crucial. As a network consists out of
nodes and connections, e-learning hould focus on strengthening the competences
of the individual and on developing connections between individuals
(communication).
− (e)Learning is becoming less important in content terms. The focus should be
more on the development of processes in learning systems and on complexity
management of the system.
− New paradigms should be defined and adopted, where learning systems are not
restricted to existing learning philosophies.

Economic changes and development

If we look at the economic development of e-learning environments we should expect
these environments to be highly valued. The old-economy based advantages of
efficiency, cost-effectiveness, the possibility of Just-In-Time learning, at the learners’
own pace and at any place should provide enough positive benefits to have a high value
to the users.
But after many experiments and five years of mainstream e-learning environments on the
market, there is still limited value for the users. Despite the launch of many projects and
pilots, there is relatively limited use.
Economic value is often expressed in share prices. Shares of in companies producing
learning environments - such as Smartforce, Digital Think, Saba and Docent - have
dropped to sometimes less than 1% of their highest value (see http://finance.yahoo.com).
It seems that economic principles are changing. Davies and Meyer (1998) describe, in
their book ‘The speed of change in a connected economy’, the way in which forces of
speed, intangibles and connectivity are challenging business behaviour. The table below
illustrates new business behaviour or “Management Mind Set”. The last column of the
table illustrates some of the implications for e-learning in SMEs.

123
 Management Mind-set (Davis & Meyer, 1998)

Product Service Offer For e-learningin SMEs

Time Time of sale Period of contract Life of Hour, day, (ir-)regular
Horizon consumer need Short learning processes life
long
Buyer Price, Ongoing support Upgrade Upgrading horizontal and
concerns delivery, ability vertical is, more important
convenience than the content
Cost focus Direct Period Design Both ways in continuously
designing
Source of Manufacturing Training, Platform Community / social /
value maintenance collective learning
Design Fixed, uniform Customised Learning Ever developing, often old
including
Constructivistic
Revenue List price Subscription Subscription + Participative role
Model period user fees
Marketing Brand loyalty Relationship Community Learning together more
objective building building important than supplier, the
electronic learning
environment and the available
content

In the new economy the roles of the producer /supplier and consumer changes
continuously and often both parties play at the same time both roles. As an example: a
producer of packaging machines sells a packaging machine to a company. He or she sells
the machine but implicitly buys information, such as the need for the machine or the
problems that occur in implementation. He also buys the networks (direct or indirect) the
company has with other potentially interested companies.
As soon as a problem occurs the consumer offers information to the supplier that can be
used for improving the machine and for providing advance improvements in services to
other users.
e-Learning takes places in such an ‘offer-environment’. Information and knowledge are
exchanged in an iterative manner between SMEs and e-learningcompanies – both those
providing learning environments and those providing learning materials and facilitation.
Support and evaluation are particularly important for SMEs. Evaluation will often take
place as an iterative process. Evaluation can allow the prediction of future scenarios and
allow steering of technology implementation and learning processes.
Another issue is that in the transition (buying/selling) process from producer to consumer
both sides use different criteria in evaluating a product. Whereas the producer looks at
stability and functionality, design, communication concepts, image and distribution, the
consumer evaluates the ‘benefit’ it has for him or her (see table below).

Producer
Consumer
Distribution Social identity
Price

Image Relation between

buyer and seller

124
 Evaluation of the offer
Market Value
Communication concept Availability
Design Desire
Functionality Benefit

Consumers of an e-learningenvironment have their own way of looking at an

environment. Some examples of the way in which perceptions are shaped are given
below.

Social identity:

A child does not evaluate the Microsoft Network (MSN) on the basis of its functionality,
but on its potential for identifying with others in his or her peer group. It is difficult to
convince consumers to participate in e-learningwhen there is no social or cultural
acceptance and practice of learning in their peer group. One of the most important factors
in determining participation in e-learningwill be the type of work and the community the
individual is in.

Relation between buyer and seller:

− Users of Lotus Notes will find it easy to use Lotus Learning Space. They are used
to the environment and can access the learning environment from within Lotus.
− If there is already a good and effective relationship with a training organisation, it
will be difficult to persuade people to use an e-learning environment, other than e-
mail for contact with the trainer, unless that training organisation itself promotes
e-learning.

Availability:

− If a trainer does not show up at a training session, learners will complain.

Similarly if a trainer is not available when needed. The same applies for e-
learning. An e-learning environment should do what it promises: Just-In-Time,
Just-Enough, at any place, pace.
− Often, e-learning courses do not match with the rhythms of the technology they
utilise. Many courses are too long and do not correspond with the learning style of
the individual or group.

Desire:

− Why do so many people enjoy playing a simple game like Tetris or Pacman?
What is driving people to use these programmes instead of beautifully designed
learning environments? It seems that many people like to absorb themselves in the
environment without having to learn new ideas.
The benefit of e-learning is different for each actor:
− For the learner: easy to learn, solving problems is easier, higher payment, status of
e-learning, etc.

125
 − For the company: cheap, effective, efficient, not loosing working time, etc.
− For a trainer: new opportunity to provide training
− For the developer: money, promotion, etc.
− For the director of a training institute: break even or make a profit.

Learning / discussion points:

− An e-learning environment should fulfil the needs of the user. Often a simple item
in a programme can be more effective than a beautiful, complete and complex
learning environment.
− Evaluation of e-learning environments will need to be a continuous iterative
process.
− Evaluation of e-learning environments will need to focus more on the criteria of
the user, than on design, functionality or cost benefit.
− New paradigms should be developed applicable to a networked society.

The context: SME environment

SMEs are like spiders in a web. To survive and to make a profit they have to optimise all
strands in the web. These include connections with suppliers, customers, the bank, the
local and regional authorities, trade associations, links to other SMEs, links between
employees.
The job of managing a SME is demanding. Often employers and employees have to fulfil
several work roles, have to work hard, be flexible, be innovative (products, production
process, marketing, selling, cost reductions, sustainability, etc.) in order to stay in the
market.
In such an environment employers and employees are in a continuous learning process. In
a natural, constructivist way they incorporate directly explicit and tacit knowledge or are
making new combinations of knowledge. They are used to acquiring and improving skills
and competences through informal learning. There is a conflict between cultures such as
technology, learning, enterprise and within this complex situation SMEs have to strive to
remain learning organisations. Generalisation will be difficult because cultures can differ
per country, region or even locality.
It is difficult to make time for formal learning during working hours. The increasing
speed of change does not allow long courses. Any spare time is needed for reflection and
developing social relations.
It is clear that for SMEs and also for larger companies the focus has changed from
products to clients. In his book “Mass-individualisation”, M. van Asseldonk (2000) says
the approach is changing from an instruction and information (product) directed
environment towards an interactive environment. The learning environment changes
accordingly and should focus more on improving competences to deal with relationships
between different actors, with innovation and with working in a changing environment.
Van Asseldonk describes the changes within mass-customisation organisations in the new
networked economy:

− From homogeneity to heterogeneity

− From prediction to responsiveness
− From efficiency to responsiveness
− From Taylorism to empowerment

126
 − Working with imbalance
− From knowledge to motivation
− From contradictions to coalitions
− From certainty to perspectives
The e-learning environment should prepare and support people working in such
environments in order for them to cope better continuous change and to create new
opportunities for the business. According to van Asseldonk an e-learningenvironment
should offer heterogeneity, responsiveness, empowerment, coalitions, perspectives and
should motivate.

But the usefulness of (or interest in) e-learning environment by enterprises appears to be
rather limited. According to a recent survey in the Netherlands, only 4% of company
sponsored training courses are on-line and according to another survey only one out of
twenty companies with an Internet connection is using e-learning. 2% say they are
considering using e-learning.
Most employers say that e-learning is not personal enough and it is more difficult to
monitor online learning. A Canadian survey (1998) showed an increase in informal
learning by adults. On average individuals learn 4 hour per week through formal courses
and learn informally 15 hours a week. Companies that invest in formal training and e-
learning environments focus more on formal training. In SMEs informal learning plays a
crucial role. Another aspect is that 50% of participants do not complete online training
programmes - 10-20% higher than ‘traditional’ training courses as an American survey
showed. It seems that online training requires self-discipline and/or learning discipline
within an organisation.
These figures have to be viewed within the context of the overall training provided by
enterprises. Another Canadian survey showed that, in 1998, 74 percent of all
establishments sponsored or provided formal or informal training for employees - 55%
sponsoring formal training. The figures for small enterprises (<20 employees) were
significantly lower, 67% and 46% respectively. Given these figures there must be
questions over the direct return on investment in the development, implementation and
use of e-learning environments for small (and medium) enterprises. e-Learning is claimed
to increase work effectiveness, but at the moment e-learning is only decreasing the time
available for work or is making the working day longer.

Learning / discussion points:

− Using e-learning in downtime or after work decreases the time people have to
exchange information and ideas, socialise and reflect.
− e-learning environments should support more informal learning processes in
SMEs.
− e-learning works best where ICT is already integrated in the working
environment.

Organisation and learning within chaos and

complexity
The choreographer, Tom Simons, has created a dance performance called “The idea of
order”.

127
He describes his dance performance like this:
“You cannot escape from the necessity to order, to organise.
Yet the manner in which you order, organise, needs to be revised continuously.
I have made fragments, which vary:
− partly or as a whole,
− either or not combined with other fragments,
− either or not related to other elements,
− different in space and time.
The many ways in which you can arrange, determines the performance.”

The learning performance in SMEs is like his dance performance. In a chaotic and
complex working environment and societal context, a person takes fragments of tacit and
explicit knowledge, competences, situations and changes in the organisation around them
and organises these in a special order, partly or as a whole, either combined or not with
other fragments, either related or not to other elements, different in space and time, and
so create an optimal learning performance and work output. This continuous process
leads to an optimal, individual work-performance and, by interaction, as a group of
individuals and, as a circle, feeds again the learning process. In e-learning we need a
choreographer in each person, organising his or her own learning performance.
Secondly, we need to make easily accessible a variety of fragments or elements, which
can be used by and are useful for the learner. Explicit knowledge (e.g. information),
learning materials, a virtual teacher and the ICT environment are only small parts of the
elements and fragments the learner needs.
Thirdly, in a complex, ever faster changing environment it is virtually impossible to
control data and facts. In a second order process of e-learning the data / information /
knowledge / competences and even the learning process are of secondary importance. It
is like preparing for an earthquake: it is impossible to create an escape route for
everybody. You never will know when it will happen and you have to take action, where
you are, according to the situation, time and place. But it is important to build self-
awareness of possibilities, of potential and alternative solutions and opportunities, for
each individual to act at that moment and place and in that context.
e-learning today is like a shelter or an escape route. We are creating or building this
instrument, but we only use it if we must and if it is the best option in a given situation or
context.
In a chaotic and complex situation, like in SMEs, it is more important to rely on
collective competence in dealing with new situations. In new forms of organisations
people rely on their networks. As a second order process they create a latent reservoir of
competences and expertise that can be used when needed.

Learning / discussion points:

− New e-learning environments should focus on supporting second order

processes and network learning processes.
− e-Learning should offer a reservoir of knowledge and information to
employees, communication between learners, and initiate and inspire people to
learn how to quickly learn and act in changing environments.

128
E-portfolio and competences
The e-portfolio is a perfect example of an instrument in a second order process. Learning
is subordinated to the process of awareness raising, to making tacit knowledge explicit
and using explicit knowledge as tacit knowledge, reflecting, initiating, inspiring, and
navigating as an instrument to account for individual competence and development.
The e-portfolio usually includes:
− an open part where (proof of) products and reflections on competences can be
stored
− a closed part where (proof of) contracts, feedback, information and private
information can be stored
Information can be ordered and presented in different ways, depending on the reason and
context in which it will be used. Data also can be used in a collective environment. It can
be used in networks to identify the competences required for a job or to identify the need
for new competences. At company level it can lead to a awareness of the supply and
demand of competences and if stored at regional or national level it offers an insight into
the status and development of existing competences and the need for stimulating (second
order process) learning processes.

Learning / discussion points:

− The e-learning environment should be closely linked to portfolio development.

− The focus must be on the competence development of a person, network,
company and / or society and should be supported by new technologies.

Learning, technology and performance in

Communities of Practice
The use of technology is not only dependent on functionality, efficiency, cost-
effectiveness and design. New approaches to the use of ICT embrace social learning and
constructivistic learning. The starting point for social learning theory is that humans are
social beings, who have competences (implicitly having knowledge) and who know
(participation, engagement). The driving force behind social theory is the act of meaning.
In a network organisation, a SME, an institute or a social gathering, a community of
practice can exist. Wenger (1998) describes social learning in his theory of Communities
of Practice (CoP)

He found that four components are essential within a CoP:

1. Meaning (within the experience)
2. Practice (by doing)
3. Community (by participating)
4. Identity (by becoming)

Failing in one of these components means less effective (a combination of affective and
effective) overall learning. Let us examine these components in an e-learning
environment.

a. Meaning
There is a gap between e-learning and meaning. What is the meaning for an
artisan or an operator to sit behind a computer to learn? What does it contribute to

129
 artisan competences? What is the meaning for a secretary to follow an online
course in Word if he or she uses only some of the features of what is taught or
when many things learned on such courses are only useful or needed much later?

b. Practice
What is the direct relationship between the e-learning environment and the work
environment? Does it follow the principle of learning by doing? Can it directly be
put in practice?

c. Community
Most people belong to a group. Learning in a community is more effective (a
combination of affective and effective) because of the social drivers behind it and
the possibility to be part of the group. Why should children in gangs or in a club
be so eager to learn and why can they learn so fast?

d. Identity
What does an e-learning environment contribute to the becoming of a person?

Wenger (http://www.ewenger.com) TIME AND SPACE

describes the thirteen fundamental 1. Presence and visibility
elements of successful communities of 2. Rhythm
practice, which can be affected by
PARTICIPATION
technology. 3. Variety of interactions
e-learning users follow this list 4. Efficiency of involvement
unconsciously. If, for instance, the user
cannot match the ‘rhythm’ of the e- VALUE CREATION
learning environment to his or her 5. Short-term value
6. Long-term value
personal rhythm and the rhythm of
work, he or she will stop the CONNECTIONS
programme. 7. Connection to the world
Another example is the learning process
of a machine operator. The e-learning IDENTITY
8. Personal identity
environment should give the machine 9. Communal identity
operator short term value (the machine
should work better, with as little as COMMUNITY MEMBERSHIP
possible downtime), should involve the 10. Belonging and relationships
operator in the total process or provide 11. Complex boundaries
learning when it is needed, and allow COMMUNITY DEVELOPMENT
operators to share experiences. 12. Evolution: maturation and integration
This list is useful for evaluating e- 13. Active community-building
learning environments, their functioning
and value. Wenger   E:   Supporting   communities 
Wenger (2001) has evaluated of practice:  a  survey  of community­
technologies supporting CoPs. He puts oriented technologies, March 2001.
forward eight different categories and
proposes the optimum for a community of practice is towards the mid point of this list.
The eight categories are:
1. Knowledge bases, where documents are stored
2. Knowledge workers’ desktop, where an ongoing integration of work and
knowledge takes place

130
 3. Project places, where work is done
4. Website communities, where social structures are formed
5. Discussion groups, where the conversation takes place
6. A place for fleeting synchronous interactions
7. e-learning spaces, where instruction takes place
8. Access to expertise, where knowledge can be exchanged.
Whilst Wenger sees e-learning as important in supporting CoP, it is most effective when
combined with other systems and approaches to the use of ICT. This is a critical issue for
those seeking to develop e-learning in SMEs.

Learning / discussion points:

− An e-learning environment should facilitate meaning, offer a place for meeting

people with common and / or complementary practices, offer a place for a
community and strengthen the personal identity.
− An e-learning environment should offer a community of practice support in
developing a knowledge base, desktop, project places, website communities,
discussion groups, synchronous interactions, instruction places and access to
expertise.

Learning psychology
Theories are built on paradigms. But new paradigms arise, existing paradigms evolve and
old paradigms reappear. Complexity management has to prepare people to use and
combine fragments and elements to create new learning environments.
Gardner has described the following ‘seven intelligences’:
1. Logical-Mathematical Intelligence
2. Kinaesthetic Intelligence
3. Visual-Spatial Intelligence
4. Musical Intelligence
5. Interpersonal Intelligence
6. Intra-personal Intelligence
Each person has his or her own combination and level of these intelligences leading to
different learning styles. This creates a new variable in the already complex environment
of learning. Each intelligence / learning style needs its own kind of technology
application.
Interpersonal intelligence is developed in pairs or small groups, where the e-learning
environment is enhancing, sharing, discussing, reflecting, supporting and working
collaboratively. Such an approach is underdeveloped for learning in SMEs. Computers
are useful for visualising complex patterns and machines help people with a high visual-
spatial intelligence.
Gardner had to add two ‘new’ intelligences:
1. Naturalist Intelligence
2. Existentialist intelligence
Contexts are often so complex and dynamic that it is difficult to follow a linear learning
process. People combine different knowledge and thus create new knowledge.
An e-learning environment should enable learners to learn using all their intelligences
and leaning styles.

Learning / discussion points:

131
 − A broad based e-learning environment makes it possible to learn in different
learning styles and change learning styles depending on the subject, the situation
and the organisation.

(Youth) behaviour
Times are changing, so is behaviour. Thirty years ago – when many of the teachers of
today were educated - there was a limited use of television and even more limited use of
ICT in the classroom. The pace of change was slower. Linear instruction was normal. At
home you played outside, were a member of a club, read a book or studied.
Now things have changed slightly. In a recent survey in the U.S. (2001) on young people
and the impact of ICT, a boy of 17 years wrote:
“I multitask every single second I am online.
At this very moment:
– I am watching TV,
– Checking my e-mail every two minutes,
– Reading a newsgroup about who shot JFK,
– Burning some music to a CD,
– And writing this message.”
He is multitasking through several different processes and is learning while multitasking.
In this survey children between 12 and 17 years old describe what they use the Internet
for. The table below shows use of the internet by those who use it daily and those who
are less frequent users.

Activity daily less

Send or receive mail 99 87
Send instant messages 89 64
Research products online 74 60
Download music 73 40
Listen to music online 70 52
Visit a chat room 62 50
Buy products online 39 26
Create a web page 34 16

Although not completely clear, it would appear that the internet is used more for social
purposes than for learning.
But whilst older people may have difficulties in using ICT for distance (informal)
learning, young people have fewer problems. As a place for meeting people, it keeps an
emotional distance. They also can be more their ‘true self’ and, by meeting people,
explore who they are. 56% of children using the Internet have multiple identities. One of
these identities is very much themselves, while others are more general (e.g. for school,
relatives), or completely fake identities. Children are even able to enter a chat room with
two or more identities and steer the discussion.
They meet strangers (60%) and hold discussions (63% of the 60%). 50% of the children
online have participated in Instant Messaging with people they have never met
In this environment young people live, learn and work.

132
What does the older generation think about this? A 15-year-old girl said:
“I wouldn’t talk about it with my parents, they’d flip out
and probably restrict my access to Internet.”
In the survey, answers from parents and children are compared.

Parents say: Kids say:

I know where my kid is going online
They often talk to kids about Internet use
Educational benefit of Internet
Kids use Internet for homework (65%)
38% of kids have e-mail account
28%: kids use Instant Messaging
They don't know
They don't
Socialising and communication
Music (57%), e-mail (56%), surfing for
fun (50%), games (48%), IM (40%),
chat (39%), homework (38%)
71% often have more than one
56% do

Our one-to-one, often linear (although often named ‘interactive’) learning approach
through present e-learning environments does not match with the behaviour ofthe new
generation.

Learning – discussion points:

− The future of the e-learning environments lies more in offering communication,

possibilities for socialisation, multitasking, and emotions / fun.
− Different phases of development of an individual need different approaches.

Technology that changes the way we learn and work

Computers are incredibly fast accurate and stupid.
People are incredibly slow, inaccurate and brilliant.
Together they are powerful beyond imagination.
Albert Einstein (1879-1955)
When looking at the development of e-learning we also have to look at the development
of ICT and the impact on working and learning. In this chapter two arbitrary examples
are described to trigger the imagination of the reader about e-learning in the future.
The first example comes from the agricultural sector. This sector is known as a
knowledge intensive sector. The first innovations took place in agriculture ten thousand
years ago and food remains one of the basics for life on earth.
After many experiments into the behaviour of animals and optimisation of the
agricultural production process, the next step is the speaking animal. In experiments it is
already possible to ‘understand’ and translate several cow-calls and distinguish between
different cows (http://www.tb.fal.de/staff/jahns/animal.htm). It seems that in the future
we need less (e)learning to learn about the behaviour of cows (if we would presume it is
possible to learn it at all). Instead we hear from the cow directly what she needs, and
could even develop a system to automatically meet her needs. Are the technology
solutions overtaking the need for e-learning?
A second example is wearable technology. Massachusetts Institute of Technology has
developed ‘intelligent clothing’ (http://www.media.mit.edu/wearables/mithril/). A person
will be able to get access to information and can communicate with other persons online
every moment and at any place through their clothes. Explicit knowledge will be readily

133
available. New technologies will arise. Systems are being developed where the fabric of
the clothing itself will be the computer, and sensors and displays and other output devices
will be integrated in glasses, lenses or earphones.

Learning / discussion points:

These learning points are really questions. There are no concrete answers for these
questions yet.
− What and how will people need to learn in technology-integrated environments in
the future?
− Which development schemes need to be followed in order to stimulate and steer
the development of future oriented environments?
− Will e-learning have a function and meaning in such an environment?

Summary
E-learning have been discussed from the point of view of nine different disciplines. The
e-learning approach up to now follows the present learning approaches in schools and at
courses. ICT only added the technology to replace certain parts of the learning system
(library ◊ database, conversation ◊ online conversation, group discussion ◊ chat, etc.).
By using the technology it is easier en more efficient to deliver, but in this article it is
questioned if this approach is leading to improving learning and converting knowledge
and activities into practice and finally in an increased sustainability and profitability (in
that order) of SMEs. It is suggested to focus more on informal learning, supporting
communities of practice and to create a continuous development process through the e-
learning environment.
Up to now e-learning environment only support or substitute existing learning
environments. Technology has improved over the last decade and is able to support a
transition from traditional learning to continuous competence development and to support
the sustainability and innovativeness of SMEs
This article doesn’t give any conclusions. It offers per chapter several learning points,
which can be useful for further discussion and for further development of effective e-
learning environments.

References
Asseldonk M van, Massa-individualisering: Maatwerk zonder meerkosten. Samson,
Deventer, the Netherlands, 2000.
Cornelis Prof Dr A, De vertraagde tijd. Essence, Middelburg, The Netherlands, 1999.
Cross J, Sources of e-learningInformation. Internet Time Group, March 2000.
(http://www.internettime.com/Learning/Information.htm#what)
Davis SM, Meyer C, BLUR: the speed of changing in the connected economy. Ernst &
Young LLP, 1998.
Dugas T, Green L, Leckie N, The impact of Technologies on Learning in the Workplace:
Final Report, Quebec Canada, March 1999.
Gibson R, Covey SR, Goldraff EM, Rethinking the future: rethinking business,
principles, competition, control, leadership, markets and the world, Nicholas Brealey
Publishers, 1998.

134
Keefe D, Dickinson D, How Technology Enhances Howard Gardner’s Eight
Intelligences. (http://www.america-tomorrow.com/ati/nhl80402.htm)
Lenhart A, Rainie L, Lewis O, Teenage life online: The rise of the instant-message
generation and the Internet’s impact on friendships and family relationships. Pew
Internet & American Life Project, Washington, June 2001.
(http://www.pewinternet.org/reports/pdfs/PIP_Teens_Report.pdf)
Linn CE, The Linn general Theory of Marketing. Meta Management AB, Stockholm,
1999, rev 2003. (http://www.metamgt.se/sidor/gtm_theory30302.pdf)
Livingstone DW, Exploring the icebergs of adult learning: Findings of the First
Canadian Survey of Informal Learning Practices. University of Toronto, Canada, 1998.
(http://www.oise.utoronto.ca/depts/sese/csew/nall/res/cjsaem.pdf)
Wenger E, Supporting communities of practice: a survey of community-oriented
technologies. North San Juan, USA, March 2001
Wenger E, Mcdermott R, Snyder WM, Cultivating communities of Practice. Harvard
Business School Press, Boston,

135
136
137