Escolar Documentos
Profissional Documentos
Cultura Documentos
COLLABORATIVE E-LEARNING
COMMUNITIES
To my friends, family
and
ii
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
And so on. In this way we try gradually to analyse all things, to put together
things which at first sight look different, with the hope that we may be able
to reduce the number of different things and thereby understand them better.
iii
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
Acknowledgments
Many other people have also provided valued input to my research through discussions,
participating in empirical work, or commenting on written work (or more than one of
these). I particularly wish to thank Sophi Danis, Sara BenIsaac, my Yoga mates,
especially Ilana Isserow, Jennifer Pearl and Lesley Todd, who have been through the
course with me; John Henderson for all the fun and support; also Catherine Spiro, Betty
Shane, Ben Daniel, Martha Christopoulou and last but not least, Mariza Smirli for their
help and support.
Special thanks to my Dad, Konstantinos, my Mum, Aphrodite and my Sister, Georgia for
being such a tolerant and supportive family.
Most of my work has been funded by the Greek Ministry of Education and Religious
Affairs given as three years of educational paid leave of absence. For this, I need to
thank Theodoros Birbas, Socratis Papathanasiou, Epaminondas Georgopoulos, and
Panayiotis Zevlas. In addition, many thanks to the Greek School Network and in
particular, Michael Paraskevas and Vangelis Grigoropoulos for their help and providing
access to research space.
Great thanks to Alexander Muir, Jenny Preece and Ben Shneiderman for the continuous
inspiration as well as their insights and energy that enabled me to overcome obstacles.
Finally, special thanks to my examiners, Dr Judy Ramsay and Professor Stephen Lerman
for the detailed comments and insights that improved this thesis.
iv
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
Declaration
v
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
Abstract
This study provides new multidisciplinary approaches for tools and evaluation
techniques to ensure quality in collaborative e-learning communities. The research
problem was the Greek teachers’ absence of participation in e-learning discussions
for 3 years. Three conceptual frameworks were used to understand and evaluate the
situation: passive and active participation, the collaborative e-learning episode, and
the sense of e-learning community index. Two interventions were made, collaborative
e-learning and the introduction of associated software-based tools: participation
graphs and avatars, MessageTag, a tool to depict the levels of critical thinking in
collaborative e-learning, and social network analysis tools, the visualisation
interactions nodes and centrality. Ethnotechnology was the research design
triangulating quantitative and qualitative data as well as social network analysis.
The originality of this study lies in the investigation of the processes on the social and
learning aspects of collaborative e-learning and associated tools.
The results indicated that the suggested frameworks and tools can be useful in
supporting collaborative e-learning communities.
The wider implications from the findings emphasise the need for: the organisations’
e-learning readiness; the e-learners’ prior knowledge and ability to interact; the
facilitation of e-learners’ social awareness; the change of teaching and learning
approaches for different levels and types of interactions, participation, and critical
thinking; the development of collaborative e-learning communities; the use of tools
anchored in learner-centred design and solid pedagogical frameworks. If all
components are present in an online course then e-learning quality can be ensured.
vi
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
Περίληψη
Η πρωτοτυπία αυτής της έρευνας έγκειται στην ερευνητική διαδικασία για την
κοινωνική και μαθησιακή διάσταση της συνεργατικής μάθησης και των σχετικών
εργαλείων. Τα αποτελέσματα έδειξαν ότι τα προτεινόμενα πλαίσια και εργαλεία
μπορεί να είναι χρήσιμα για την υποστήριξη των διαδικτυακών μαθησιακών
κοινοτήτων.
vii
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
Table of Contents
Acknowledgments …………………………………………………………… iv
Declaration ……………………………………………………………………. v
Abstract ……………………………………………………………………….. vi
Περίληψη (Abstract in Greek) ……………………………………………… vii
Table of Contents …………………………………………………………… viii
Index of Tables ……………………………………………………………… xi
Index of Figures …………………………………………………………….. xiii
Index of Graphs …………………………………………………………….. xiv
Index of Appendices ……………………………………………………….. xv
viii
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
ix
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
x
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
Index of Tables
Chapter 1
Table 1.5-1. Chapters Overview ……………………………………………………… 11
Chapter 2
Table 2.2.1-1. The Socially-Shared Cognition Approach ………………………….. 21
Table 2.3.2-1. Reasons for lurking in e-learning communities ................................ 29
Table 2.3.2-2. Strategies for enhancing activity in e-learning .................................. 29
Table 2.4.2-1. Instructional Design problems …………………………………………. 36
Table2.5.1.1-1. Design principles for online and e-learning communities ................ 42
Table 2.5.2.1-1. Learning and Instructional Activities ……………………………….. 46
Table 2.5.2.1-2. MessageForum Collaborative Learning Attributes ………………… 48
Table 2.5.2.1-3. Similarities and differences between MessageForum & InterLock 50
Table 2.6.2-1. Levels of participation measurement …………………………………. 53
Table 2.6.2-2. Levels of activity measurement ………………………………………... 54
Chapter 3
Table 3.2-1. Research Design ………………………………………………………….. 70
Table 3.2.2.1-1. Collaborative e-Learning Episodes Coding Matrix ………………… 78
Table 3.2.4-1. Observations, interventions and evaluation ………………………… 84
Table 3.2.4-2. Limitations and strengths in Time-Short Series …………………….. 84
Table 3.2.4-1. Research methodology ……………………………............................ 85
Chapter 4
Table 4.3.1.1-1. Courses categories and number of e-learners ………………… 100
Table 4.3.1.1-2. e-Learners’ posts and views in the 6 active courses …………….. 103
Table 4.3.1.1-3. e-Learners’ replies and dates ………………………………………. 104
Table 4.3.1.1-4. E-Learning Engineering for Moodle@GSN ……………………….. 106
Chapter 5
Table 5.2.-1. Iterative Design Blocks ………………………………………………… 110
Table 5.2.2.1-1. Use of MessageTag …………………………………………………. 123
Table 5.2.2.3-1. Research Design …………………………………………………… 129
Table 5.2.3-1. Demographics for the three Greek Teachers / Moodle Developers 130
Table 5.2.3-2. Questionnaire open questions …………………………………….…. 131
Table 5.2.3-3. Tools Pedagogical Usability Scores …………………………………. 132
Chapter 6
Table 6.1.1-1. The questionnaires’ selection process ……………………………… 139
Table 6.2.3-1. Greek teachers’ knowledge and attitudes on collaboration
and participation ……………………………………………………… 145
Table 6.3.1-1. Moodle@GSN forums and users view log files ………………….. 147
Table 6.3.1-2. Forums and users view log files in the research pool …………… 148
Table 6.3.1-3. Total number of posts ……………………………………………… 148
Table 6.3.1-4. Temporal overview of all activities ………………………………… 149
Table 6.3.1-5. Temporal overview of posted messages (add post/forum) …….. 150
Table 6.3.2.1-1. Number of active and passive participants …………………….. 151
Table 6.3.2.2-1. Active participation levels (Initial proposition) ………………….. 152
Table 6.3.2.2-2. Active participation levels (Second proposition) ………………. 153
Table 6.3.2.3-1. Passive Participation Levels …………………………………….. 154
xi
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
xii
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
Index of Figures
Chapter 2
Figure 2.2.1-1 The Eyeball of Participation ………………………………………… 22
Figure 2.5-1. Google trend history for online community and e-learning …………. 39
Figure 2.5-2. PLATO III ………………………………………………………………… 40
Figure 2.5.2-1: Categorisation of collaborative learning tools ................................. 45
Figure 2.5.2.1-1. InterLock Interface ………………………………………………..…. 47
Figure 2.5.2.1-2. MessageForum Attributes in a Discussion Topic ………………… 49
Figure 2.6.1-1. The Collaborative E-learning Episode (CeLE) ……………………….52
Figure 2.6.2-1. Participation Levels in Collaborative e-Learning Communities …… 54
Chapter 3
Figure 3.2.1.3-1 Ethnotechnology and methods ……………………………………... 75
Figure 3.2.3-1. Questionnaire Design Methodology ………………………………… 82
Chapter 4
Figure 4.2-1. Organisation of the Education System in Greece 2003/04 …………. 96
Figure 4.3.1-1. Moodle@GSN research context …………………………………….. 99
Chapter 5
Figure 5.2.1.1-1. Visualisation Interaction Tools (VIT) Nodes & Centrality …........ 111
Figure 5.2.1.1-2. Visualisation Interactions Tools (VIT) production line ………….. 111
Figure 5.2.1.2-1. Course and individual participation levels graph ………………. 112
Figure 5.2.1.2-2. Participation evaluation graphs production line …………………. 113
Figure 5.2.1.2-1. Initial Design for Message Tagging ………………………………. 114
Figure 5.2.2-1 Total Codes Network in the Web Design pool ............................ 117
Figure 5.2.2.1-1. Participants and number of messages (ATLAS.ti) ……………… 118
Figure 5.2.2.1-2. Location of VIT on the discussion forum …………………………. 119
Figure 5.2.2.1-3. VIT Nodes …………………………………………………………… 119
Figure 5.2.2.1-4. VIT Centrality ………………………………………………………... 119
Figure 5.2.2.1-5. MessageTag …………………………………………………………. 122
Figure 5.2.2.2-1. Lurkers overall view in VIT Centrality (right) ……………………. 124
Figure 5.2.2.2-2. Redesign of participation graphs …………………………………. 126
Figure 5.2.2.2-3 CeLE MessageTag tool ……………………………………………… 127
Figure 5.2.3.1-1. Discussion on tools’ Greek names ……………………………….. 134
Figure 5.2.3.1-2. Pedagogical Usability Attributes ................................................ 135
Chapter 6
Figure 6.1.1-1. The e-tutors in the online course ……………………………………. 138
Figure 6.1.1-2. Normality overview for tools pedagogical usability and utility
in HCE ……………………………………………………………….. 140
Figure 6.2.1-1. Participants’ location in Greece …………………………………… 142
Figure 6.5.7-1. GSN adjacency matrix in UCINET ……………………………….. 185
Figure 6.6.2-1. Correlations analysis in HCE 3.0 ………………………………… 204
xiii
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
Index of Graphs
Chapter 4
Graph 4.3.1.1-1. Online course categories in Moodle@GSN ……………………… 101
Graph 4.3.1.1-2. Comparison between number of messages and replies ............. 102
Graph 4.3.1.1-3. Activity in the online course …….…………………………………. 104
Chapter 5
Graph 5.2.2-1. Participating countries ……………………………………………….. 115
Graph 5.2.2.1-1. 31/03/2006 – 5/28 e-learners ……………………………………… 121
Graph 5.2.2.1-2. 04/04/2006 – 10/28 e-learners …………………………………….. 121
Graph 5.2.2.1-3. 08/04/2006 – 10/28 e-learners …………………………………….. 121
Chapter 6
Graph 6.2.2-1. Correlations between time in education, use of computers,
and Learning Management Systems (LMS) …………………….. 143
Graph 6.2.3-1. Communication with the educational authorities …………………. 146
Graph 6.3.1-1. Comparison between sent messages and
messages for analysis ................................................................. 149
Graph 6.3.1-2. Logs of overall activity VS posting ………………………………… 150
Graph 6.3.2.3-1. Active & Passive Participation locus
from the same participants ....................................................... 155
Graph 6.4.1-1. Comparison between messages for analysis, richness of text,
And discussion depth ……………………………………………. 159
Graph 6.4.2-1. CeLEs factors’ comparison graph …………………………………. 162
Graph 6.4.2-2. Comparison for number of words posted by e-learners
and e-tutors ……………………………………………………….. 164
Graph 6.5.1-1. Community evolution elements ……………………………………. 167
Graph 6.5.1-2. New members’ contributions ………………………………………… 170
Graph 6.5.1-3. Roles in the e-learning community ……………………………….. 172
Graph 6.5.2-1. Participants’ opinions on e-learning community elements …….. 173
Graph 6.5.2-2. Comparison of themes and community elements ………………. 174
Graph 6.5.3-1. Scatter plot for empathy …………………………………………… 175
Graph 6.5.5-1. Passive & Active Participation Process …………………………. 178
Graph 6.5.6-1. Elements that show community evolution: e-learning …………. 182
Graph 6.5.6-3. The e-learning facilitators …………………………………………. 183
Graph 6.5.6-2. Correlations between codes on collaborative e-learning quality 184
Graph 6.5.7.1-1. Reciprocal ties in GSN (a) & the research pool (b) ………….. 187
Graph 6.5.7.1-2. Structural equivalence dendrogrammes in GSN (all) ……….. 170
Graph 6.5.7.1-3. Structural equivalence dendrogrammes in the
research pool (all) …………………………………………… 170
Graph 6.5.7.1-4. Structural equivalence dendrogramme in GSN (e-learners) .. 191
Graph 6.5.7.1-5. Structural equivalence dendrogramme in the
research pool (e-learners) ………………………………… 191
Graph 6.5.7.3-1: VIT Nodes in CeLE IX ………………………………………….. 198
Graph 6.5.7.3-2. VIT Centrality in CeLE IX ………………………………………. 199
xiv
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
Index of Appendixes
xv
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities
xvi
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 1: Introduction
Introduction:
Chapter 1 introduces the research context, the research problem and the aims and
objectives of this study. Because socio-cultural learning theories and e-learning
design techniques evolved separately, they lack convergence. This is one of the
causes of quality problems in e-learning. An example is presented from the Greek
teachers’ e-learning community project aimed at their professional training and
development. An initial study showed that the mere provision of information did not
facilitate new-knowledge construction. Consequently, this study targets the
development of evaluation techniques and associated tools to enhance participation
in collaborative e-learning communities.
1.1 INTRODUCTION
For the past 50 years two main trends have been observed in education, the
socio-cultural focus and the use of technology. However, these two trends have
evolved almost separately. Socio-technical design and user-centred design were
planning approaches aiming to acknowledge that the development of interactive
technologies increasingly relies on an appreciation of the social circumstances in
which systems are used. Educational or instructional design is the systematic
processing of activities to solve an instructional problem with the aid of technologies.
Nonetheless, educational design and in particular e-learning design neglected the
dual and situated persona of the learner; she acts as both a user and a learner. In
addition, the e-learning systems were found to be information-based mainly
supporting monologue instead of being communication-based towards dialogue. For
this reason they fail to support e-learners’ transition between internalisation to
externalisation and becoming active participants. Thus, mere provision of information
points to poor e-learning quality. So, if educational design could understand the
technology of collaborative practice, e-learning quality could be improved.
This chapter introduces the concept of quality in e-learning and examines its
relationship to socio-cultural collaborative learning and associated design. The
research context is the Greek teachers’ e-learning community, started in 2003 as part
of a project for online teachers’ training and aimed at enabling teachers to acquire
new competencies. However, these aims were not met because of passive
participation and this implies that information acquisition may not be automatically
related to collaborative learning. It appears that learning within e-learning
communities is not always successful.
ICT is the backbone of the knowledge economy and has been recognised as an
effective tool for promoting economic growth and development (World Bank Report,
Chen & Kee, 2005). Despite the expansion of ICT, general access to ICT varies
across continents, and countries even within the same continent, indicating a digital
divide (Reddy & Manjulika, 2002). Organisations, educational institutions and
business have been investing in the use of ICT in Education, or in what ESRC now
calls Technology Enhanced Learning (TEL) (ESRC, 2006). E-Learning is a
component of TEL and describes learning via the Internet, intranet, and extranet (WR
Hambrecht and Co, 2000:8). The freedom that e-learning offers and the increasing
number of online courses provided by educational organisations offer new
opportunities for personal and professional development in a life-long learning
course. Nevertheless, teachers’ education has been severely criticized on the
grounds of both quantity and quality (e.g. Perraton & Potashnik, 1997; Darling-
Hammond & Bransford, 2005; Thompson & Schmidt, 2007).
To solve this problem, UNESCO suggests that countries need to keep pace with
technological development and the changing competencies, reflected in the
curriculum and teacher training. Economic advantage will accrue to a population that
acquires competencies in processing information into knowledge and applying it in
work and everyday life. These competencies are not only related to using the devices
but also working on procedures that give access to information and skilfully
transforming information into knowledge. As this is the task of the educator,
educational systems will become a national resource as important as the traditional
factors of production-land, labour, and capital. This in turn would cause educators to
become more important, their productivity and their wages should increase, but they
can also expect the nature of their jobs to change with a great deal of specialization
(UNESCO, 2002:633-640). To support this life-long learning context, companies and
institutions often use commercial Learning Management Systems (LMS) for online
teachers’ training. Commercial LMS like Centra, and Blackboard, and Open Source
software as Bodington, Dokeos or Moodle are nowadays widely used; for example,
there are more than 26,124 registered sites from 182 countries, 1,855 in the U.K.
(http://moodle.org/sites/, last access 29/05/2007), and 97 in Greece, one of them is in
the service provided by the Greek School Network (GSN).
1
Leonardo DaVinci project ‘Implementing Standards for European e-Tutor Training’ (ISEeTT,
http://www.etutorportal.net/). ISEeTT aims to define the core curriculum and quality standards for European e-Tutor
training in relation to different national contexts. The Greek partner is the Educational Research and Evaluation
Group, Foundation for Research Technology - Hellas (http://www.forth.gr/).
2
There was no additional information for this project other than the website: http://fecone.passionforlearning.eu/.
However, Moodle@GSN appears not to have worked in that there has been a high
level of passive participation, that is absence of posting, for more than three years
(1077 days on the 13/10/2006 according to the log files). Although the Greek
teachers do visit and download material as apparent from the log files, there is no
evidence that the Greek teachers use the e-learning service effectively and skilfully to
transform information into knowledge. Following one of the participants in one of the
studies “absence of participation in GSN is probably because (desire, time, money
for) planning and organisation are lacking. Collaboration in design and planning by all
stakeholders to bring pedagogy and technology together is lacking as the most
valuable idea/suggestion: learner / user involvement in the design”. It also appears
that broad collaboration is now crucial for contemporary organisations.
Even though e-learning has delivered mixed results, some advantages have been
reported in using e-learning for teachers’ training (Golian, 2000): e-learning is
individualised and self-paced; there are more opportunities to access learning
resources; e-learning is based on activities and experience (active and experiential
learning) within groups and communities (collaborative learning); time and cost are
less because of the use of the electronic form of resources; and communication is
nonlinear. Nevertheless, there are several obstacles: institutional, instructional,
technical, and personal (Piotrowski & Vodanovich, 2000).
In general, quality refers to fitness for purpose, and in this context as applied to
learning (Stephenson, 2005). A survey on e-learning quality for CEDEFOP, the
European Agency for Vocational Training, showed that 61% of the 433 respondents
rated the overall e-learning quality somewhat negatively, as ‘fair’ or ‘poor (Massy,
2002:3). The European Foundation for Quality in eLearning (EFQUEL) conducted a
European survey between 15 August 2004 and 15 November 2004 (Panorama
Report, Ehlers, et al., 2005). In this survey, 5,023 people called up the questionnaire,
of whom 28 % actually completed it, and a further 7% finished the two basic sections
on quality in e-learning. According to the results, quality relates to obtaining the best
learning achievements (50%) and ‘something that is excellent in performance’ (19%).
In brief, the PANORAMA report revealed the importance of e-learning quality: the
need for critical awareness; the need for specific analytical frameworks as the
respondents although believed that they knew about quality they showed a general
lack of information; and the need for quality requirements in e-learning design. The
researchers stressed that ‘learners must play a key part in determining the quality of
e-learning services’ and insisted on the involvement of all e-learning participants in
quality design and development by 2010 (p.11).
This means that collaborative learning is related to co-creativity and has the potential
to occur online. Therefore, interventions to support social interactions, collaborative
learning and associated tools may influence e-learning quality. It also suggests that if
one of the elements is missing then e-learning quality may be impaired.
The study translates into two sets of questions in order to study the conditions to
tackle e-learning quality. The first set of research questions will be investigated in the
literature review. These are:
Q2. Is there an educational design approach that can ensure quality in specific
e-learning contexts such as the Greek teachers’ community?
Q2.A. What is design?
Q3.B. Are there any design principles for specific educational contexts?
Q3. Are there any tools and techniques that can be used to facilitate the
formation of CeLC? If so, what are the effective characteristics and usage of
these tools?
Q3.A. Are tools and techniques for evaluating participation helpful to enable
participation?
Q3.B. Are tools for observing and analysing interactions helpful to enable
participation?
The second set of questions has an exploratory nature and will be based on the
previous results:
2. Educational Design
Q2.Ex1. In what ways can core design principles be integrated in the process
of educational design?
Q2.Ex2. In what ways can quality by design be achieved for the Greek e-
learning communities?
3. Tools to Support CeLC
Q3.Ex1. Are tools for evaluating participation helpful to enable participation?
Q3.Ex2. Are tools for structuring collaborative learning helpful to enable
participation?
Q3.Ex3. Are tools for observing and analysing interactions helpful to enable
participation in the Greek e-learning community?
Q3.Ex4. Are tools for observing reporting information on interactions helpful in
enabling participation in the Greek e-learning community?
Chapters Description
Motivation, research problem
Aims and objectives
1 Introduction
Research questions
Chapters overview
Collaborative e-learning
E-Learning communities and participation
2 Literature review Tools and evaluation techniques
Design
Conceptual framework
3 Research design Research methodologies
Understanding the research context
4 Research context Ethnotechnological inputs
Preliminary studies
5 Tools and evaluation techniques Planning and design
Implementation of previous conceptual framework
6 Main study Findings
Discussion
Summary of the Thesis
Conclusions
7 Conclusions
Recommendations
Future trends
REFERENCES
Barbera, E. (2004). Quality in virtual education environments. British Journal of
Educational Technology, 35(1), 13-20.
Berners-Lee, T. (2007). Berners-Lee warns of changes ahead. Computing Magazine.
Retrieved 17/06/2007, from
http://www.computing.co.uk/computing/analysis/2186086/berners-lee-warns-
changes-ahead.
Blackboard (n.d.). Retrieved 15/02/2006, from http://www.blackboard.com/.
Bodington (n.d.). Retrieved 15/02/2006, from http://www.bodington.org/.
CENTRA Education & Training (n.d.). Retrieved 15/02/2006, from
http://www.centra.org.uk/.
CEDEFOP (n.d.). Cedefop: The European Centre for the Development of Vocational
Training. Retrieved 21/08/07, from http://www.cedefop.europa.eu/.
Chen, D.H.C. & Kee, H.L. (2005). A Model on Knowledge and Endogenous Growth
World Bank Policy Research Working Paper 3539. Retrieved 22/10/2006,
from http://info.worldbank.org/etools/library/latestversion.asp?135703.
Daniels J.S. (2002) “Foreword” in Information and Communication Technology in
Education – A Curriculum for Schools and Programme for Teacher
Development. Paris: UNESCO
Darling-Hammond, L. & Bransford, J. (2005). Preparing Teachers for a Changing
World: What Teachers Should Learn and be Able to Do. San Francisco, CA:
Jossey-Bass.
Delich, P. (2006). Pedagogical and interface modifications: What instructors change
after teaching online. Published doctoral dissertation, Pepperdine University,
Malibu, CA. Retrieved 10/09/2007, from
http://proquest.umi.com/pqdlink?Ver=1&Exp=09-09-
2012&FMT=7&DID=1144195631&RQT=309&attempt=1.
Dokeos (n.d.) Retrieved 15/02/2006, from http://www.dokeos.com/.
Downes, S. (2005). E-learning 2.0. eLearn Magazine. Association for Computing
Machinery, Inc. Retrieved 12/04/2006, from
http://elearnmag.org/subpage.cfm?section=articles&article=29-1
EFQUEL (n.d.). European Foundation for Quality in eLearning. Retrieved
20/08/2007, from http://www.qualityfoundation.org/.
Ehlers, U., Hildebrandt, B., Görtz, L. & Pawlowski J. (2005).Quality in European e-
learning – PANORAMA report. A study by the European Quality Observatory.
Retrieved 12/11/2006, from http://cms.eun.org/shared/data/pdf/quality_in_e-
learning_panorama_en.pdf.
ESRC (2006). Economic and Social Research council. Retrieved 13/05/2006, from
http://www.esrcsocietytoday.ac.uk/ESRCInfoCentre/opportunities/current_fun
ding_opportunities/tel_call.aspx?ComponentId=14262&SourcePageId=5964.
Franklin, S., Peat, M., Lewis, A., & Sims, R. (2001). Technology at the cutting edge:
A large scale evaluation of the effectiveness of educational resources. Paper
presented at ED-MEDIA, the 13th Annual World Conference on Educational
Multimedia, Hypermedia & Telecommunications. 25-30 June 2001, Tampere,
Finland.
Garrett, R. (2004). The real story behind the failure of the UK eUniversity. Educause
Quarterly, 27(4), 3-6.
Golian, L. M. (2000), ‘Utilizing Internet Resources by Education Professionals in the
New Millennium’, Information Technology and Libraries, 19(3): 136-143.
GSN (n.d.). The Greek School Network. http://www.sch.gr/en/.
Hartnell-Young, E. & McGuinness, K. (2006). Evaluation of an Online Community:
Australia's National Quality Schooling Framework. In Niki Lambropoulos and
Panayiotis Zaphiris (Eds) (2006). User-Centred Design of E-Learning
Communities. Hershey, USA: Idea Group.
Literature Review:
• How and why educational research moved from the study of the
individual to the study of the community
• New propositions
2.1 INTRODUCTION
change and for this reason, they did not acknowledge their passive participation
(Illeris, 2002). Nonetheless, Mezirow proposed that dialogue, the collaborative
learning pillar, was the solution.
Teasley and Roschelle provided a clear distinction anchored in the idea that
tasks are divided between participants: “each person is responsible for a portion
of the problem solving”. However, such division is not deliberately required from
the participants, although roles exist naturally as a spontaneous division of
labour. In their review, Borgers and Baranauskas (2003) advocated collaborative
learning as it had more advantages than other types of group learning. It
empowered and enabled learners to solve problems and understand subjects
more easily since discussing ideas and constructing arguments through dialogue
could shape in-depth learning. Collaboration is an interactive process that
engages two or more participants working together to achieve outcomes they
could not accomplish independently (Salmons & Wilson, 2008). UNESCO’s
definition embraces most of the aforementioned concepts; collaborative learning
occurs
Thus shared creativity for new knowledge building is the ultimate collaboration
learning target that can also be expanded to online settings. Working on shared
tasks implies learners’ participation and engagement.
However, some e-learners do not actively participate but lurk 1 . According to the
Free Online Dictionary for Computing (http://foldoc.org, added on 14/06/1997),
lurking is a messaging jargon for activity of one of the "silent majority" in an
electronic forum; it is posting occasionally or not at all, but reading the group's
postings regularly. This term was not pejorative; for example, reading the
Frequently Asked Questions was recommended netiquette for beginners who
needed to learn about the history and practises of the group before posting. This
distinction of passive and active participation appeared in computer-supported
tasks; the participant who controls the mouse tended to be "executor", while the
other was likely to be the "reflector" (Blaye et al., 1991). According to Miyake
(1986:174): "The person who has more to say about the current topic takes the
task-doer's role, while the other becomes an observer, monitoring the situation.
The observer can contribute by criticising and giving topic-divergent motions,
which are not the primary roles of the task-doer." Miyake referred to active as
well as reflective learning since criticism could support new knowledge
production.
If passive and active modes are acceptable, there should be some processes
and pedagogical methodologies to create and maintain the transition between
these. Rather than the facilitator or educator providing rules for learning (Berge &
Collins, n.d.; Wegerif, et al., 1998:495), learners took responsibility for their own
learning, as, according to Mercer (1995), they had to: talk to create the context;
engage in collaborative learning activities; create common knowledge; and follow
ground rules that encourage the exchange of relevant ideas and active
participation. Initial intention is another prerequisite, important enough to form a
1
Passive participation and lurking will be used interchangeably in this thesis as a demand for changing the term
lurker was observed in the literature as well as online communities and conferences (e.g. E-mint community,
the JISC, Joint Information Systems Committee online conference discussions on ‘Innovating E-Learning’,
http://www.jisc.ac.uk/elp_conference07).
cooperative principle needed before interactions occur (Grice, 1975). This idea
was further developed in psychology as the principle of the “least collaborative
effort” (Clarke & Wilkes-Gibbs, 1986:26), as an active concern for the
construction of mutual understanding in order to develop a shared focus and a
shared narrative (Crook, 1994:176). In other words, it is the e-learners’ shared
goals and generated context that make collaborative learning occur; then,
collaborative e-learning continuity creates the shared narrative that fabricates its
history and builds a successful e-learning community. Prerequisites also refer to
a joint interactive space for grounding, as interactions intended to create
common ground (Clark et al, 1983), mutual understanding, knowledge, beliefs,
assumptions or presuppositions (Baker et al., 1999; Mäkitalo et al., 2001) or
“repairing” misunderstandings (Dillenbourg, 1999). Therefore grounding is built
on sociability. This means that sociability creates the initial conditions for
collaborative learning.
whether and under what circumstances collaborative learning was more effective
than learning alone. Because collaborative learning is inherently complex, it was
almost impossible to establish causal links between the conditions and the
effects of collaboration. Therefore, Dillenbourg and colleagues indicated the need
for new tools and methods for observing and analysing interactions to increase
understanding of the collaborative learning social mode. This development of an
understanding of learning is briefly presented next.
Wenger, 1991). Here, the learning process occurred within a larger physical and
social context of interactions and culturally constructed tools and meanings.
Social and individual were not different levels of study but inexorably
interconnected; this created two distinct traditions of situated cognition, one
focused on the individual and a second focused on community (Wilson & Myers,
2000). The community focus is next.
There were several community definitions; for example, Peck (1987) stated that
“If we are going to use the word [community] meaningfully we must restrict it to a
group of individuals who have learned how to communicate honestly with each
other, whose relationships go deeper than their masks of composure, and who
have developed some significant commitment to "rejoice together, mourn
together," and to "delight in each other, make others' conditions our own." The
development of communication channels, trust, support, and a sense of
belonging seemed to be significant to help a community to emerge. Lazlo and
Lazlo (1997, 2000) described the community as a group of two or more
individuals with a shared identity and a common purpose, committed to the joint
creation of meaning. The authors echoed Lave and Wenger in Communities of
Practice (CoP) (1991); CoP members shared the characteristics of joint
enterprise, mutual engagement and shared repertoire to clarify, define, and
evolve practices (Wenger, 1998). One implication of the social reproduction of
CoP was that the sustained participation of newcomers, as they become old-
timers, must involve conflict between the forces that support processes of
learning and those that work against them. This was because learning,
transformation, and change were always mutually present. Thus, CoPs were
engaged in the generative processes of producing their own future. Learning in
communities was configured through the process of becoming a full participant in
practice and being able to get involved in new activities, perform new tasks and
functions, and master new understandings.
newcomers were located at the first level, as potential contributors, and were not
voiceless or powerless but a vital part of the community. Overall, Lave and
Wenger proposed that LPP was a descriptor of engagement in social practice
that entailed learning as an integral constituent. In fact, central and peripheral
participation should not be on different levels after all (1991:35). The next table
describes CoP, the research focus, the key tenets, the research methods and
results (Table 2.2.1-1):
Research
Research Focus Key Tenets Results
Methods
-Learning is situated -Inter - -There are
-The individual, the and grounded in disciplinary different types
social and the everyday actions. research of
physical is one -The situation needs -Real settings communities.
research context. to be studied as a -Ethnographic -Conflicts may
whole. inputs allow
Learning is a -CoP characteristics: resolutions.
process of joint enterprise, -The
participating in mutual engagement, conditions that
CoP. shared repertoire facilitate
-Legitimate participation:
Research Peripheral -management
Questions Participation of the
increases in participation
What is learning engagement, process
within a complexity and -accessibility
community? responsibility. to information,
-Learner’s &
Is learning via participation in the -tools that
interactions with community lies in the enable
other individuals concept of “becoming members’
more efficient than part of the participation
learning alone? community. and learning.
-Lurkers are
What are the types legitimate participants
of social and community
engagement that members.
facilitate learning? -Tools are part of the
community’s heritage
What are the social and cultural life.
conditions that
facilitate learning?
It appears that the research focus involved the individual, the social as well as
their situated context where the individual participates in collaborative practices.
For this reason, multidisciplinary methods needed to be evolved to investigate
learning on an individual and social level as well as the tools and the
environment of participation. Thus, the sense of belonging appeared to be the
driving force for collaborative learning enhanced by tools.
The participants were located in four levels, based on the “numeric amount” of
posting to the community (number of messages). With a direction from the
periphery to the centre, lurkers were the participants who did not exhibit any
activity. On the second level there were the members who occasionally
contributed to the community. The participants and key contributors were located
on the third and fourth level. The “eyeball of participation” provided a structure to
better understand legitimate peripheral participation. McDonald and colleagues
stressed the fact that the active members are the ones who “add value” and fill
the gaps for all members in order to sustain the community.
2
Empathy is a “complex psychological inference in which observation, memory, knowledge and
reasoning are combined to yield insights into the thoughts and feelings of others” (Ickes, 1997:2)
2004; Preece & Ghozati, 2001; Lambropoulos, 2005a, 2005b). Nonnecke and
Preece (2000:127) found that some lurkers felt a sense of community, especially
when the dialogue engenders “a sense of trust and care”; this was an indirect
way to become active contributors. Lambropoulos found that the members who
developed empathy became active participants (2005a). Ramachandran
suggested that empathy was a cognitive activity triggered by the mirror neurons
(2000) and appeared to be a key in human communication regardless of the
medium used.
Research on e-learning communities appeared in the mid 80s (e.g. Hiltz, 1985).
However, Barab and Duffy (2000) argued that e-learning environments were
practice fields rather than authentic Communities of Practice (CoP) because the
activities were not real; they were educational and not part of the communities’
authentic work. On the other hand, being a participant in a community was an
essential component of the educational process (Goodfellow 2003): ‘communities
of practice’ differ from ‘communities of learners’ in that the latter are reflexively
concerned with learning whereas the former are concerned with practice, of
which learning is a corollary. (p. 3). It appears that there is a difference between
online CoP and e-learning communities. Active and passive behaviour is not a
new phenomenon; an experiment on active and passive reading was conducted
by Janis and King in 1953 (cited in Hovland et al.1953). Two experimental groups
of college students worked as ‘active participants’, who delivered talks in a group
situation; and ‘passive controls’, who read and listened to the source material.
The results indicated that all members had the potential to become active
participants; in other words, they were located in the grey zone of potential active
participation. This grey zone was characterised by the “sleeper effect” as a
change of behaviour after a lapse of time (Hovland et al., 1953). If all passive
participants have the potential to become active this requires an effort on their
behalf.
The Greek teachers belong to an off-line CoP; however, this community is not
reflected in an online CoP. For this reason passive participation has been
identified as the key variance in this study. Thus, the Greek teachers remained
on the first level of participation, that is passive participation. This variance is
imperative to be controlled on a pedagogical and operational level. The question
that arises is, if the Greek teachers do belong in an offline CoP why is it so
important to force them in active participation in collaborative e-learning
communities?
Participation in CoP involves the use of tools; artefacts used within a cultural
practice carry a substantial portion of that practice's heritage. Thus,
understanding the technology of practice is more than learning to use the tools; it
is a way to connect with the history of practice and participate more directly in its
cultural life (Lave & Wenger, 1991:100-102). In the age of ubiquitous computing
and the social phenomenon of the Web, and with the growth of what Berners-Lee
(2007), calls collaboration and creativity, communities will continue to evolve.
Thus, the question is not why it is important to engage the Greek teachers in e-
learning, but finding ways to enable it.
Posting and interactivity have come under research scrutiny early in online
research (e.g. Rafaeli, 1986; Bruckman & Resnick, 1995) where interactivity was
related to dependency among messages (Rafaeli & Sudweeks, 1997). Figures
related to interactivity and passive participation are different, depending on the
dynamics of the media, and result in specific lurking behaviours (e.g. Fish et al.,
1990; Whittaker et al., 1998; Monthienvichienchai, 2004). Passive participation
can be tracked approximately using different methods provided that the software
supports it, for example counting the views (e.g. Nonnecke & Preece, 2000) or
using proprietary tools that log lurkers’ communicative behaviour (e.g. Soroka, et
al., 2003). Carroll & Rosson (1996) referred to lurkers-to-posters ratios 100:1,
Sproull and Faraj (1997) reported an 80% of lurking, Preece from 46% to 82%
(2000) and Lambropoulos, working with a Greek teachers’ online community,
99% (2002). Lurkers in one community can actively participate in other
communities where they believe they have something important to say (Klemm,
1998).
Passive participation was observed for discussion forums or blogs (Williams &
Jacobs, 2004); the relationship between writing and reading behaviour (Ebner &
Holzinger, 2005); if activity influences learning efficiency (Beaudoin, 2002); or
underlying assumptions of e-learning pedagogy (Gulati, 2006). In particular,
Beaudoin (2002) found that almost 23 out of 55 (42%) of online education
master’s degree students, preferred to read other learners’ messages. A
questionnaire sent via email to 23 "low visibility or "no visibility" learners (p. 150),
showed that e-learners were: reading assignments and others’ comments;
conducted web searches; writing assignments; and spent least time on writing
comments for online discussions. Other observed behaviours were: connecting
(visiting the community); browsing (passively participating in community’s life);
attending time durations; contributing opinions; responding to specific posts; and
interacting (responding in a reciprocal manner) (Rafaeli et al., 2004).
reasons may be related to reasons inherent in the teaching and learning modes
(e.g. Klemm, 1998; Earley & Gibson 2002). Klemm (1998) advocated that there
were psychological and social reasons with roots in the passive conditioning and
the “entertain me” mode of mass media. Both learners and teachers had been
exposed to television and traditional classroom teaching that deprived them of
critical thinking. In particular, Klemm, working on an online conferences context,
believed that lurking restricts creative thinking and teachers foster this behaviour
with their practices. To Khine and colleagues (2003) there is trainee teachers’
inability to actively participate in online discussions; the participants were not
critical thinkers and failed to sustain interaction. E-learners have been found
reluctant to criticise each other (Hughes & Daykin, 2002:222) even if they are
“forced” to be active (e.g. Williams, 2002; Oliver & Shaw, 2003). Greeks are not
different; in fact, Buhayer (2005) referred to the Greek media as transmitting a
model of a reluctance to follow the stories through, lack of investigation and
questioning that killed conversation in Greek homes (p. 157-158).
-e-tutors need to know what they are looking for and involve
themselves to help make it happen
-peer grading
-set of institutional norms promoting trust,
-promoting knowledge sharing as a norm of the organization,
-employees are trusted,
Archichvili
-sharing is a moral obligation.
et al.
-building multiple face-to-face CoP
(2003)
-clear norms
-standards for sharing knowledge.
-obtain active participation
-have transparent interface,
Swan et.
-importance attached to the role of the e-tutor,
al., (2003)
-valued and dynamic discussion
Rafaeli et. -familiarity with the community
al., (2004) -develop a sense of community
The next section will investigate whether current design supports them.
Online Education is in its infancy and has yet to construct design models
that will address the problem of the learner as a user (Wallace, 1999; Smulders,
2003). Learning how to be a user is about understanding the technology rather
than learning to use the tools (Lave & Wenger, 1991:100-102). There is an
assumption that the educational systems are easy to use by the educators and
learners. Users in educational settings act as both users and learners and design
has to support their dual persona (Smulders, 2003). However, technologists tend
to build techno-centric systems for use by academics, this means integrating
several levels of functionality which is geared towards the e-tutors rather than
considering the e-learning participants. For example, besides the Greek teachers
and e-tutors, the Greek Schools Network (GSN) consist of developers, engineers
and decision makers. Additionally, today’s users are typically multi tasking; this is
especially so for younger users who might do homework whilst listening to Mp3s
and chatting with friends (Dede, 2005). In order to support multi tasking the
question of the interface becomes important since a usable system, one with an
easy to use interface, will enable users to concentrate on their tasks. Thus,
interface design might become pivotal and make the difference between a
system that is used and one that isn’t. Design may influence participation quality.
Many design definitions exist; for example, design underpins every form
of creation from objects such as chairs to the way we plan and execute our lives
(Dini, 2005). But to Suchman, design developed strategies and tools insensitive
to particular circumstances (1987:121) because plans actually derive after the
completion of the course. Such a post-hoc design structure challenged the
foundation of computational design as a linear process of development. It
appears that the design of new systems is always problematic.
iterative design, and (iv) integrated design. When systems did not operate as
expected then user feedback should be sought and used in the redesign as
systems should conform to expectations. User-centred studies provided more
coherent frameworks and models for analysis of what is going on and what
should be done to ensure users’ satisfaction. Shackel (1991) suggested that
usability could be measured by examining learnability, effectiveness, attitude and
flexibility; Nielsen (1993) thought it was efficiency, learnability, memorability,
errors and whether the system is subjectively pleasing. The International
Organisation for Standardisation (ISO 9241-11, 1998) defined usability as a
measure of the quality of user’s experience when interacting with a system, in
terms of effectiveness, efficiency, and satisfaction. Measurements and
assessment procedures had to ensure the product met the purpose of design.
Overall, educational systems and their integrated tools constitute the means by
which learners are permitted or restricted in the use of their learning capabilities.
Instructional design and engineering are design processes to ensure their
educational fulfilment.
solutions. ID begins with initialization and project planning (how the instructional
design is carried out), the design and development phase (appropriate strategies
and approaches in targeted contexts), and a quality assurance phase focused on
deployment, evaluation, and assessment. It also includes easy-to-use checklists
and presentation tips offering a comprehensive insight in ID and deployment
outside the control room. This interdisciplinary approach involved all stakeholders
in design by covering all stakeholders’ benefits defined under specific criteria:
roles, skills, characteristics, activities, commitments, and responsibilities. As with
the previous design approaches, identification of intents and planning based on
stakeholders’ goals, skills, background and characteristics provided the design
backbone. Fenrich said that his model works in ideal situations; in reality
instructional designers need to adjust the plan to given situations by integrating
evaluation results into system requirements and re-design specifications. This
means that Fenrich implicitly agreed with Suchman on the post-hoc nature of
design.
However, it was suggested that systematic ID models had been accused of not
reflecting actual practice and not supporting new competencies for the 21st
century and post-industrial societies (Table 2.4.2-1):
It appears that the major problem is related to the ID process itself with regard to
its implementation and flexibility. Some other problems related to e-learning
quality and recorded in the Panorama report (Ehlers, et al., 2005) were caused
by: e-learning participants’ unfamiliarity with the design process; division between
‘academic’ and ‘corporate’ approaches; lack of awareness of the need for quality
standards; the dual identity of the learner as a user was neglected; stakeholders’
engagement in the early stages of design was neglected; and the fact that design
in artificial settings makes it vulnerable to the Hawthorne effect, so little evidence
existed as to how to use technology effectively.
These problems were relevant to the situated nature of the learning environment
as they have a “conclusion-oriented” nature instead of a “decision-oriented” one.
In addition, social interactions were partially neglected; with the Web 2.0
signpost, instructional designers and engineers foresee the need of an e-learning
2.0 stage (Downes, 2006; Karrer, 2006; Jennings, 2005). At this point in time,
educational design needs to consider the social and collaborative structure of e-
learning communities for learners’ content creation. The question that arises is
not why ID and IE suffer from so many problems, but to what extent designers
need a more flexible or a more systematic model. Depending on the
organisational target, a managemental educational change suggests a more
open model, whereas design for particular systems requires a more coherent
model. According to Leigh (1998), instructional designers need to focus either on
one aspect of learning and instruction and act as consultants or matter experts.
Since the field is becoming too broad for most designers to work with authority,
For active learning, point and click is not enough (Klemm, 1998).
Dillenbourg (2000) stressed the social aspect of e-learning as a designed
information space where learners are actors i.e. they co-construct the information
space. Kollock (1998) emphasised the need for such social computing: ‘The key
challenges the Internet community will face in the future are not simply
technological, but also sociological: the challenges of social interaction and social
organization.’. In other words, e-learning applications need to consider social
interactions between e-learning participants. So far, there are different types of e-
learning tools categorised under several criteria: temporal, as in synchronous,
asynchronous activities or both; the medium of communication as in text, audio,
visual, simulation and their combinations; the direction of activity as in broadcast,
email, messaging or interactive; web-based as offline or online; and the privacy
levels, these are public or private. The computer-based collaborative learning
tools are divided into conversation and collaboration tools. (For reviews, see
Conrad & Donaldson, 2004; Jonassen, n.d.; Kommers et al., 1992) Jonassen
said there was a chasm to consider between tools that support information
provision and groupware.
The quest for tools in the e-learning history goes back to the 60’s. (For a detailed
view, see Moodle http://docs.moodle.org/en/Online_Learning_History, and
wikipedia http://en.wikipedia.org/wiki/History_of_virtual_learning_environments.)
Systems to support e-learning communities were not included in associated
reviews because they were considered as a “third place”, a room for social
collaboration outside home and work (Bruckman & Resnick, 1995). They were
also excluded from the history of online communities by Ambrozek and
colleagues (2004), and had a reference of 33 words in Preece and colleagues
(2000). The following graph from Google Trends depicts the separate routes of
the two terms in the literature (Figure 2.5-1):
Figure 2.5-1. Google trend history for online community and e-learning
The separate routes meet just before 2007 suggesting a trend within e-learning
communities. Nonetheless, the social aspects of collaborative e-learning were
apparent from the early years of the educational technology research. It is
interesting to observe that the word “collaborative” exists in early designs.
In his review of e-learning software developments over the last 30 years, Rumble
(2001) proposed four models of teaching and learning: the transmission, the
constructivist, the socio-cultural, and the metacognitive model. Following
Rumble’s review, current LMS can be located only on the first three levels. In
other words, higher order thinking is not supported by current LMS. For example,
Moodle, related to this research, was based on socio-constructivist theories
(http://docs.moodle.org/en/Philosophy). Today, Moodle developers have
synthesized Rumble’s first three concepts in the “social constructionist
pedagogy”:
3
Figure retrieved from the blog http://siliconuser.com/?q=node/12 - Posted June 8th, 2007 by Joshua Coventry
(Permission to use the image in the Thesis was acquired via email on 06/12/2007.)
Despite the fact that the Moodle developers built on sound pedagogical
approaches, Moodle does not explicitly support collaborative e-learning
communities. Overall, it appears that current learning management systems have
not evolved significantly, compared to the pace of emerging technologies of
social networks and user-generated context.
To Kim, success is related to both the social and textual infrastructure for
situated activity and learner-created context. It also seems that more than ten
years after MediaMoo, a design to support socio-cultural e-learning is still an
issue.
It appears that most principles for online and e-learning communities are related
to community management and targeted to promote group coherence and
resolve differences as well as supporting members’ individuality. The e-learning
community guidelines for synthesizing pedagogy, community management, and
socio-technical design were suggested by the writer (Lambropoulos, 2006).
Other more informal principles to support teachers’ e-learning stressed the need
for a socio-cultural basis and came from the teachers’ community TappedIn®;
Gray and Koch (n.d.) suggested that the use of familiar settings, support of social
facilitation, and allowing flexible grouping supports e-learning communities.
Design factors for collaborative e-learning communities appear to be complex
and interrelated.
Kim (2001) suggested that there is no single answer for developing principles for
e-learning communities. However, one solution may be that the principles should
be open and agile enough so to be incorporated in any situated e-learning
interface design.
levels scheme that did not allow critical thinking (Nassaji & Wells, 2000). (For a
review on a state of art technology for supporting collaborative learning see
Jermann et al., 2001.)
It appears that there are two kinds of tools to support human-human and
human-computer interactions: tools for observing and tools for analysing
interactions, divided in three categories, social/cognitive, cognitive/metacognitive,
and task/communicative (Figure 2.5.2-1):
Collaborative Learning
Tools
(Dillenbourg et al., 1996)
Observing Analysing
Interactions Interactions
Jonassen said that each attribute should represent an instructional activity. There
were different tools supporting collaborative learning discourse such as the
collaborative notebook (O’Neil & Gomez, 1994) or the knowledge integration
environment (Bell, Davis & Linn, 1995) based on utterances as particular kinds of
speech acts (initiate, continue, repair and acknowledge), sentence openers and
floor control (I agree, Could you explain etc) (Traum, 1994). Another approach
that will be explored in detail in the next chapter was introduced by Mercer (1995)
and Wegerif and colleagues (1998); the authors worked on a collaborative
learning model for disputational, cumulative, and exploratory talk. Baker (2000)
suggested that what was required for tools to structure dialogical sequences was
relations between theory, model and corpus (i.e. transcriptions of interactions
data), and then design legitimate objects. Two examples of such models
translated into collaborative e-learning tools were InterLock and Messageforum.
InterLock was the by-product of McAlister’s PhD project Academic Talk (2004)
(http://learning.north.londonmet.ac.uk/ltri/academictalk/). As with AcademicTalk,
InterLock, worked on the same attributes to support collaborative e-learning
argumentation (http://www.interloc.org/; last accessed 21/12/2006), and Mercer
and Wegerif, participated in the new project (e.g. Ravenscroft, Wegerif, Mercer &
Hartley, 2006). InterLock viewed collaborative e-learning argumentation as a
game that promoted engagement and learning (Figure 2.5.2.1-1):
The next example had a more flexible nature and structured collaborative e-
learning in discussion forums. MessageForum was built by Jeong (2005) about
the same time as InterLock (2006) to support computer-supported collaborative
argumentation. Jeong suggested that online dialogical argumentation lacked
depth and was redundant. To solve these problems he created the
ForumManager, a tool to support collaborative e-learning argumentation based
on Toulmin (1958). The ForumManager was an MS Excel application for
downloading and analyzing messages (and message texts) in Blackboard
threaded discussion forums using Internet Explorer browser. This discussion
analysis tool created reports in Excel including reports on social interactions built
on social network analysis, that is visualisation of e-learners’ interactions. Table
2.5.2.1-2 describes the collaborative e-learning attributes in detail using symbols
and attributes rather than message openers:
The translation of the above model in a tool is depicted next (Figure 2.5.2.1-2):
As Dillenbourg and colleagues had predicted in 1996, Jeong found that the
visibility of the structure helped learners’ reflection; more replies were elaborated
on previous ideas; there were greater gains in knowledge acquisition; there were
fewer unsupported claims, greater knowledge of argumentation processes; there
was no difference in knowledge acquisition, application of domain content, and
convergence towards consensus; and lastly, there were fewer challenges per
argument. The findings about the effects of using message constraints and
message labelling were similar to studies on other projects such as
ShadowPDForum (Jonassen & Ramirez 2005); NegotiationTool (Beers, 2004);
Future Learning Environment 3 (FLE3) (Leinonen et al., 2003); and Computer-
Supported Collaborative Learning Environment (CSILE) (Scardamalia et al.,
1994).
According to Jeong, the tool supported the visibility of the mechanisms that
facilitated three main areas: the strategic uses of message labels; the potential
group performance problems associated with personality traits; and methods and
tools used with message labelling to diagnose problems. Lastly, one of Jeong’s
recommendations for future research was to integrate tools in the forums for real-
time feedback to optimize group performance. One of the consequences of not
having the results in real-time is lack of situated social presence. Jeong has
presented similar results from other studies (e.g. Jeong & Davidson-Shivers,
2006; Jeong, 2008). Jeong’s tool also produced social network analysis (SNA)
graphs (sociogrammes) via Microsoft Excel (not in real time). Since SNA tools
appeared in online community research after 2000, it seems that Dillenbourg and
colleagues (1996) may have predicted another e-learning trend. The similarities
and differences between InterLock and ForumManager were (Table 2.5.2.1-3):
MessageForum VS InterLock
+ n/a Initial information
Attributes
- n/a Agree
ARG = Question
ARG = Challenge
BUT = Reason
EXPL = Reason
EVID / Maintain
The differences as regards the interface are mainly based on the different ways
in which the implementation of the main concepts have been carried out.
InterLock is a system whereas MessageForum is a tool incorporated in e-
learning systems such as Blackboard. The differences were connected to
interface usability as well as the visibility of attributes structure. Overall, it
appears that any tools to support collaborative e-learning need to aid:
The question that arises is whether conceptual frameworks can aid the design
process. This will be discussed next.
Based on the literature review, there are three propositions for design, the
Collaborative e-Learning Episodes, the Participation Levels, and the Sense of the
e-Learning Community Index.
4
Examples of CELEs can be found on Appendix A_VII_13 and Appendix X.
Inform Explain
Interaction Pedagogical
Value
Inform Suggestion
Explain Elaboration
Question Reflection
Evaluate Question
Explore Elaboration
Evaluate Reflection
Explore
The CeLE is proposed to indicate the collaborative e-learning levels, and as such
its value in real-time. This means that all e-learning participants can instantly be
aware of the level and the value of participation, as soon as the learner describes
the conversation act in an accurate way. Secondly, CeLE can identify the actors
and the modes of preferred participation. Lastly, CeLE, in combination with other
tools, can support the e-learning community’s socio-dynamics, that is the type of
interactions that can foster the community’s creativity. So if a CeLE can be
translated into a tool it will support observing and analysing collaborative e-
learning interactions as it:
X Sense of Community
4 High 76-100%
3 Medium 26-75%
2 Low 1-25%
1 Zero
The above levels show the symmetry/asymmetry in participation and also depict
the grey zone between passive and active participation characterised by the
sleeper effect; the latter is the white area where the participants decide to make
the first step, make the least collaborative effort. Furthermore, there is an area
outside the taxonomy and in the middle of the participation eye that refers to the
sense of belonging to the community and does not depend on active
participation. Active participation is initiated with the very first message and has
three levels, low, medium and high. If these levels can be displayed in real-time
on a discussion topic level and a course level, then participants in e-learning will
be able to be proactive by managing and evaluating participation in real time.
Real time online community management and evaluation of participation
provides immediate information delivery as needed for all stakeholders. It is cost-
effective; it is iterative rather than one-off as results are concentrated on the
process; it minimizes risk as decision making depends on facts rather than
assumptions; it is a catalyst for the stakeholders to accomplish their goals faster.
REFERENCES
Ambrozek, J., Bundesen, L., & Cothrel, J. (2004). Online Communities in
Business: Past Progress, Future Directions. Paper presented at
International Conference on Virtual Communities 2004. Retrieved
15/12/2006, from http://www.infonortics.com/vc/vc04/slides/cothrel.pdf.
Anderson, B. (1997). Work, Ethnography and System Design. In A. Kent & J. G.
Williams (Eds.), The Encyclopedia of Microcomputers (Vol. 20, pp. 159-
183). New York, NY: Marcel Dekker.
Archichvili, A., Page, V., & Wentling, T. (2003). Motivation and Barriers to
Participation in Virtual knowledge-sharing Communities of Practice.
Journal of Knowledge Management, 7(1), 64-77.
Argyle, M. (1991). Cooperation: The basis of sociability. London: Routledge.
Axelrod, R. (1984). The Evolution of Cooperation. New York, NY: Basic Books.
Baker, M. J. (2000). The Roles of Models in Artificial Intelligence and Education
Research: A Prospective View. International Journal of Artificial
Intelligence in Education, 11, 122-143.
Baker, M.J., Hansen, T., Joiner, R., & Traum, D. (1999). The role of grounding in
collaborative learning tasks. In P. Dillenbourg (Ed.), Collaborative
learning. Cognitive and computational approaches (pp. 31--63).
Amsterdam: Pergamon.
Barab, S., & Duffy, T. (2000). From Practice Fields to Communities of Practice. In
D. Jonassen & S. Land (Eds.), Theoretical Foundations of Learning
Environments (pp. 25-55). Mahwah, NJ: Lawrence Erlhbaum Associates.
Beaudoin, M. F. (2002). Learning or Lurking? Tracking the 'Invisible' Online
Student. The Internet and Higher Education, 5(2), 147-155.
Beers, P. J., Boshuizen, H. P. A., & Kirschner, P. (2004). Computer support for
knowledge construction in collaborative learning environments. Paper
presented at the AERA'04 - American Educational Research Association,
Learning in innovative learning environments Symposium, San Diego, Ca.
Bell, P. L. (1997). Using argument representations to make thinking visible for
individuals and groups. Paper presented at the Computer Support for
Collaborative Learning, Toronto, Canada. Retrieved 02/12/2005, from
http://www.oise.utoronto.ca/cscl/papers/bell.pdf
Bell, P., Davis, E. A., & Linn, M. C. (1995). The knowledge integration
environment: Theory and design. Paper presented at the CSCL'95 -
Computer Supported Collaborative Learning Conference, Bloomington,
IN. Mahwah, NJ: Lawrence Erlbaum Associates, 14-21.
Berners-Lee, T. (2007). Berners-Lee warns of changes ahead. Computing
Magazine. Retrieved 17/06/2007, from
http://www.computing.co.uk/computing/analysis/2186086/berners-lee-
warns-changes-ahead.
Berge, Z. L., & Collins, M. (n.d.). E-moderators. Retrieved 23/04/2005, from
http://emoderators.com/.
Berger, C., & Kam, R. (1996). Educational Software Design and Authoring.
Retrieved 24/05/2005, from http://www.umich.edu/~ed626/626.html.
Bichelmeyer, B. A. (2005). The ADDIE "Model" - A Metaphor for the Lack of
Clarity in the Field of IDT. IDT Record. Retrieved 25/06/2005, from
http://www.indiana.edu/idt/shortpapers/documents/aect2004.htm.
Blaye, A., Light, P. H., Joiner, R., & Sheldon, S. (1991). Joint planning and
problem solving on a computer-based task. British Journal of
Developmental Psychology, 9, 471-483.
Bloom, B. S. (1956). Taxonomy of Educational Objectives, the classification of
educational goals – Handbook I: Cognitive Domain. New York, NY:
McKay.
3.1. INTRODUCTION
Sasse, referring to the different aspects of the HCI nature, says that “the lack of a
single agreed research strategy discipline leaves an individual researcher planning a
specific research undertaking out on a limb” (n.d.). This multifaceted nature is
reflected in the lack of a single agreed strategy; however, this disadvantage can be
tackled by triangulation. Nonetheless, there is a risk; some times an amalgamation of
different research methods fails because of the disengagement between methods
and their underlying methodology (Boehner et al., 2007).
This chapter builds on situated e-research. The first section discusses issues for e-
learning research and ethics. Then, it employs an aspect of ethnography, that is
ethnotechnology, to examine the research context. Ethnotechnology is used
alongside human-computer and human-human interaction analysis to shed more
light on collaborative e-learning communities by triangulating sides of space and
time. Time-series design is suggested as a means of coordinating e-research.
the final study was hosted and wiped out all database tables; on a second occasion,
at the researcher’s request, the system operators collected data including
demographic details for GSN’s own use and also for research purposes.
On the other hand, Robinson and colleagues (2007) suggest that a combination of
methods is needed, but in order to avoid any form of control over the participants,
data can be collected as it occurs naturally. To Robinson and colleagues, working on
ethnographically-informed empirical studies, it is legitimate to consider such data in
order to understand practice in its own terms. This approach may resolve the
problem in research created by the Hawthorn effect. However, in reality “there are no
clearly defined criteria for appropriate ethical behaviour for all researchers or all
research activity” (Anderson & Kanuka, 2003:58).
In order to address the ethical issues, this study was approved by three different
organisations: the London South Bank University Ethics Committee, the Greek
Pedagogical Institute, and the Innovations Office, part of the Greek Ministry of
Education and Religious Affairs. Prior to the transfer of this research to LSBU there
had been no approval sought for the two initial studies not presented in this thesis.
Nonetheless, these were conducted under all ethical considerations and principles
and participants’ consent was obtained for all studies. The consent was obtained by
the participants’ signature on a pro-forma which emphasised that they could withdraw
from the study at any time without providing any explanation. The participants were
allowed to keep a copy; in addition, the right to withdraw was stressed in the user
policy (netiquette) available online as well as sent to the participants prior to the
study.
RESEARCH DESIGN
Target Use
Research Methodologies Implications
Coordination Evaluation
for Design
1 HCI-HHI Ethnotechnology √ √ √
2 Qualitative Methodologies
HHI A Message analysis √
3 HHI Social network analysis √
4 Quantitative Methodologies
HCI-HHI A Questionnaires √ √
HCI B Log Analysis √ √
5 n/a Time-series design √
The research design targets methodological triangulation, that is the application and
combination of several research methodologies of the same phenomenon to
corroborate one set of findings with another; the hope is that two or more sets of
findings will converge on a single proposition (for a review of triangulation, see
Massey, 1999). Initially, ethnotechnology can increase understanding of the research
context for human-human and human-computer interactions (HHI & HCI).
For the purpose of this study, the examination of research design should
scrutinise the situation as regards participation in collaborative e-learning
communities and determine implications for designing successful collaborative e-
learning communities. Ethnographic research was found to represent a long tradition
of studying social processes in real life situations, used to uncover the knowledge,
ideas, beliefs, values, and purposes of systems use, and thus informing the
designers about the setting. It has been exploited in computing research, especially
after Suchman (1987) discussed the profound mismatch between the generic models
of work on which IT systems are built at the time and the actual nature of work in
which they are used.
3.2.1.1. Ethnography
there was a problem related to finding common ground and communication channels
for both the fieldworker and the engineer. Despite the efforts to ease their differences
(e.g. Hughes et al., 1997), and after almost 10 years of research, Hughes (1999)
concluded that there are major differences between ethnography and systems
design; they are two different mentalities.
A more recent approach considers that an engineer is able to conduct the fieldwork
herself. Although this approach has created questions on the engineer’s
epistemological validity there were successful implementations (Furnis, 2004; Sharp
et al., 2006). Studying teamwork in their own agile software development groups,
Sharp and colleagues (e.g. 2005, 2006) adopted ethnographic perspectives in order
to put design to work in particular contexts, adopted and adapted by people in the
course of practice, contradicting the definition of the “user” as a passive recipient of
technology. Thus, design is the active process of incorporation and co-evolution of
technologies, practices, and settings.
Assumptions on the use of ethnography for the ethnographer’s role as well as the
locus of e-research were revisited. If the designer/researcher is located within their
domain of study, in this case her own community of practice, can immerse herself
easily since the prolonged engagement and immersion in the context required in
ethnography already exists. Thus, she can progressively reconfigure new ways to
understand ‘users’ and ‘user contexts’ by understanding work practices within her
own culture and society as inspirations and foundations for design activity in order to
support new ways of working. These were the assumptions that were used in this
study.
Ethnotechnology, has been only recently used to shape a strategy in computing and
e-learning research as in collaborative e-learning (Guribye & Wasson, 2002; Guribye,
2005) and online Communities of Practice (CoP) (Paiva, 2005; Talamo, 2005;
Talamo & Ligorio, 2002). It is a specific field for studying the impact of technology
raised from the observation of a mismatch among the users’ way of tools’
implementation and the functions for which the developers had planned them
(Grossen et al., 2006). Ethnotechnology has been recently employed for developing,
animating, and analysing online CoP in the ITCOLE EU project (Talamo, 2005;
Talamo & Ligorio, 2002). The project aimed at the creation of software tools,
pedagogical best-practices, and testing, based on cycles of software development.
Talamo and Ligorio combined ethnotechnology and discourse analysis in
synchronous and asynchronous communication within the ITCOLE CSCL system,
built to support collaborative e-learning communities. Following Talamo,
ethnotechnology can be supported by textual analysis and can rely on
ethnotechnologist’s empathic understanding of her context, abandoning her
preconceptions, and treating everything as ‘strange’. Full participation is feasible
when she is fully engaged as when they belong to a CoP, however, her research role
is partly covert.
The research issues are similar to ethnography, such as the role of the researcher
and lack of generalisability since ethnographic approaches are always in principle
incomplete (Hine, 2005).
The GSN Deputy Director, Michael Paraskevas, said that the Greek teachers
were not using the tools provided. His observation brings forward one of the primary
aims of ethnotechnology, studying the impact of technology raised from the
observation of a mismatch among the users’ way of tools’ implementation and the
functions for which the developers had planned them. Conducting scenic fieldwork
within the Greek Community of Practice (CoP) appears to have both a real and
virtual locus. Participant observation in activities such as seminars and conferences,
talking to colleagues, and conducting interviews with individuals from the Greek
educational authorities can reveal some aspects of the problem of participation in
collaborative e-learning communities. As for the online environment, focus groups as
well as the quantitative human-computer (logging) and qualitative human-human
interaction analysis (messages analysis) can shed more light in understanding the
Greek e-learners. Thus ethnotechnology can be utilised in e-learning design.
Focus groups, widely used in Human-Computer Interaction (e.g. Faulkner, 2000), are
compatible with ethnography and thus ethnotechnology. Suter (2000) worked on the
dilemma of their use in ethnography and provided three reasons they should be
seriously considered as an alternative way for generating data: (a) access to
participants' interaction on topics that are either difficult to observe or rare in
occurrence; (b) a focus group improves ethnographic practice by providing another
option for generating appropriate data; and (c) the method raises important questions
of relevance between focus groups and participant observation. Provided that the
participants in a focus group discuss the topic of their specialisation and interest, a
focus group can provide insights in hidden aspects of design with data that could not
be acquired otherwise. So in this study, one focus group is used to reveal best
practices and tools for activating the passive participants and a second to evaluate
the tools and evaluation techniques before their implementation.
Logging can also offer opportunities for correlations in order to identify practices that
work better than others.
ETHNOTECHNOLOGY
Questionnaires Logging
Open & closed questions Log files
(qualitative & quantitative data) (quantitative data)
Fieldwork
observation, interviews
(qualitative data)
In this study, CeLE analytical framework was found to neglect significant information
related to social interactions; these were the social cues such as words at the
beginning and end of a message (e.g. greetings and sign offs) and emoticons. Thus
two more analytical frameworks were considered in order to increase understanding
of collaborative e-learning; Henri’s (1992) five dimensions for message analysis, and
Fahy and colleagues’ Transcript Analysis Tool (TAT) (2001, 2005). Her model
provided an initial framework for coding CMC discussions; however, it lacked
detailed criteria for systematic and robust classification of electronic discourse
(Howell-Richardson & Mellar, 1996). In addition, it has been criticised for poor
theoretical support and being strictly a teacher-centred instructional paradigm
(Gunawardena & Lowe, 1997). Despite these limitations, there is an advantage in
Henri’s proposition, which is the social cues coding for analysing social messages:
self-introduction, expression of feeling, greeting or emoticons for describing emotions
(Henri, 1991:126). After Henri, more researchers have attempted to analyze such
social effects of conferencing exchange (e.g. Rice & Love, 1987; Walther, 1996) or
sociability of online communities (Preece, 2000).
As a result, one CeLE was the unit of coding as the most basic meaningful segment.
The attributes of a CeLE were initially based on conceptual and empirical work;
however, inter-rater reliability had to be reached to ensure correct coding. Therefore,
an analysis of 25% of the data was conducted by an independent researcher in
Atlas-ti™ based on the CeLE codes without any prior discussion. This revealed an
initial reliability of 3% on the data which was clearly unacceptable. The classification
indicators were rewritten in a clearer and simpler way and a further 10% of the data
was coded giving a reliability of 50%. Then, the themes and clusters were discussed
and analysed with the independent researcher giving a reliability of 90% on a further
25% of the data. A second independent researcher verified the process with a
reliability of 93% on a 25% sample. Reliability in the 90%+ range was operationally
considered as being acceptable.
The schema has been also tested by 2 colleagues using messages from different
units of analysis from SKYPE and chats (Lambropoulos et al., 2008). It appeared that
social cues can be related to building social interactions prior to collaborative e-
learning and can be included in coding sub-units within initiation (Table 3.2.2.1-1):
Quantifying qualitative data was essential in thematic analysis. Other than counting
the words in a post as an indicator of text richness, two relevant concepts were found
in Fahy and colleagues’ research on their Transcript Analysis Tool (TAT) (2001,
2005), density and intensity. Density was the ratio of the actual number of
connections observed, to the total potential number of possible connections: Density
= 2a/N(N-1), where "a" was the number of observed interactions between
participants, and "N" was the total number of participants. The researchers stressed
that the measure of density is sensitive to the size of the network, so larger groups
will be likely to exhibit lower density ratios than will smaller groups.
Intensity in this study referred to the levels of participation and persistence where
persistence is the level to which participants pursue topics. Persistence was related
to interactivity and discussion depth operationalised by measuring the number of
levels of communication in a particular discussion thread from the first posting to the
last (depth of discussion threads).
Other than the posts’ quantitative and qualitative examination, investigations of social
interactions can shed more light in the research context and can triangulate the
findings with ethnotechnological fieldwork.
Cohesion: Network density for group thickness, reciprocity, cliques, and structural
equivalence were used to measure the level of cohesion. Network density is the
proportion of possible links in network that actually exist; it was evaluated by the
adjacency connection reports. Sent-Received (S-R) number of messages is related
to participants’ reciprocity. More specifically, reciprocity is the number of ties that are
involved in reciprocal relations relative to the total number of actual ties (Hanneman
& Riddle, 2005). A clique is a set of actors with each being connected to each other
in smaller groups. Structural equivalence and in particular the CONCOR technique
(CONvergence of iterated CORrelations, White, Boorman, Breiger, 1976; Breiger et
al., 1975), describes the actors that have similar relations to others in the network
with dendrogrammes. So the degree to which two nodes are structurally equivalent
can be evaluated by measuring the degree to which their columns are identical:
Centrality: Group centrality (Everett, 2005; Freeman, 1979) refers to the distribution
of power between the community members and is measured by centrality, closeness
and betweenness. In this study it referred to the total number of Sent-Received
Messages (direct links), out-degree (replies made) and in-degree (received
messages) centrality. Group closeness is defined by the normalised inverse sum of
distances from the group to a node outside the group (Everett, 2005:61); in this study
closeness was related to reciprocal distances. Betweeness is the number of indirect
links in which the actor is required as an intermediary; this characterise the mediator
as the controller of the information flow in a network.
SNA Tools: SNA has been used for offline (e.g. Breiger, 2004; Bender-deMoll &
McFarland, n.d.), off-line and online (e.g. Wellman, 2001), and online educational
contexts (e.g. Daniel, 2007; Laghos & Zaphiris, in press). Presenting relationships
and perceiving solutions derived from visualisation can assist annotations,
consultancy or revision since data visualisation proved to be important for user locus
of awareness, control and initiative (Shneiderman, 2000). However, even though
SNA tools can depict the activity in online human-human interactions, the use of
these tools in educational contexts is offline. Offline SNA tools need the researcher to
input the data either based on observation (e.g. Petropoulou, 2006; Laghos &
Zaphiris, in press) or extract the data in a file such as excel files (Jeong, 2005).
These data are inserted to SNA software for visualisations graphs, however,
providing an out-of-date evaluation and decision making.
SNA tools could support e-learning discussions in real time. Kreijns and colleagues
(2002) have suggested that if a group awareness widget existed, a software tool
providing the learner group awareness about the others in the task and in the non-
task context can enhance groups’ sociability. Additionally, social presence and co-
presence can enhance the sense of community by providing a picture of the
community (Beer et al., 2005). Social presence was the degree by which a person
was perceived as real in an online conversation (Meyer, 2002:59). Real-time SNA
has started to support e-research; for example, Microsoft uses SNA tools to visualise
users’ clicks on web pages (Milic-Frayling, 2007). To date, some open source offline
SNA software used in discussion research are SoNIA (Stanford University; Bender-
deMoll & McFarland, 2006), Pajek (University of Ljubljana, Slovenia; Nooy et al.,
2005), UCINET (Analytictech, http://www.analytictech.com/download_products.htm)
and JUNG (Java Universal Network/Graph Framework, http://jung.sourceforge.net/).
JUNG was found to be compatible to Moodle and was integrated in Moodle for the
purposes of this study. Other than the SNA online use, an SNA software is essential
for data analysis. After studying comparisons between several SNA desktop
applications (Laghos, 2007; Huisman & van Duijin, 2005) UCINET (Borgatti et al.,
2002) was found most suitable for this study. The main reasons were its usability and
the support provided (Everett and Borgatti; personal communication via email,
September 2007).
Overall, fieldwork and thematic analysis provide the data for qualitative analysis that
can be quantified for statistical analysis; logging and Social Network Analysis provide
data for quantitative analysis and social networks visualisation. Open and closed
questions in a questionnaire could be used for triangulation so as to reveal whether
two or more sets of findings could converge in a single proposition.
2007); revealing lurkers’ opinions (e.g. Nonnecke, 2000; Gulati, 2006); or evaluation
of evaluation tool’s effectiveness (e.g. Silius et al, 2003).
Three questionnaires were given to the participants, at the beginning, in the middle
and in the end of study. The first questionnaire was used to acquire information on
demographics, conditions of working and learning on the internet and initial
Lastly, the final section presents the e-research design needed for the e-research
coordination.
Time settings refer to two main sets, defining the baseline(s), and time series.
Baseline refers to the observation of behaviour prior to any treatment designed to
alter behaviour. As such, the treatment effect is demonstrated by a discontinuity in
the pattern of pre-treatment and post-treatment responses. The groups which are
going to be used in this study are inactive. The latter suggests a solid baseline for
treatments and effects related to causal inference, not affected by threats like history,
natural development and maturity for studies mostly observed in children’s research.
In time-short-series design aggregation and causal inference are not necessarily
affected if a detailed amount of data could be collected. There are three dimensions
to be investigated in order to examine the nature of intervention: (a) the form of the
effect (the level, slop, variance and cyclicity); (b) its permanence (continuous or
discontinuous) and (c) its immediacy (immediate or delayed).
The initial baseline describes the research context before any intervention. Then
community management and tools (treatments) support the e-learning community.
Limitations and strengths of time-short-series design in this study are (Sanson-
Fisher, 2004) (Table 3.2.4-2):
Time-short-series
Limitations Strengths
• Process-based framework creates close examination
• Fewer study units of both units and interventions as well as causal
limits generalisability inference
• Measures must be • All units could get intervention if it is effective
suitable for repeated • New theories can be created
use
• Flexibility related to individuals, small and large units
• Depends upon • More intervention research and knowledge
successful, temporal • Can examine each intervention component
relationship between
• Interdisciplinary Research exists
intervention and
• Clear research design and data analysis
measure
• Consisted with decision making processes
ETHNOTECHNOLOGY DESCRIPTION
Secondly, additional constraints reflect the nature and circumstances under which
the research is carried out including the time frame, having one individual performing
all research functions and simultaneously being an active participant in the Greek
educational community, use of open source tools, not being an expert ethnographer,
and having limited programming skills.
Lastly, even though there are recent studies on participation in online communities,
there is no similar research for comparison in the Greek and international research
community.
REFERENCES
Anderson, T., & Kanuka, H. (2003). E-research: Methods, strategies, and issues.
Boston, MA: Allyn & Bacon.
Andrews, D., Nonnecke, B., & Preece, J. (2003). Electronic survey methodology: A
case study in reaching hard to involve Internet Users. International Journal of
Human-Computer Interaction, 16(2), 185-210.
Baroudi, J. J., Olson, M. H., & Ives, B. (1986). An Empirical Study of the Impact of
User Involvement on System Usage and Information Satisfaction.
Communications of the ACM, 29(3), 232-238.
Beer, M. D., Slack, F., & Armitt, G. (2005). Collaboration and Teamwork: Immersion
and Presence in an Online Learning Environment. Information Systems
Frontiers, 7(1), 27-37.
Bender-deMoll, S., & McFarland, D. A. (2006). The Art and Science of Dynamic
Network Visualization. Journal of Social Structure, 7(2). Retrieved
15/12/2006, http://www.stanford.edu/group/sonia/papers/DNV_JOSS.pdf.
Bender-deMoll, S., & McFarland, D. (n.d.). Interaction, Time, and Motion: Animating
Social Networks with SoNIA. Report, Stanford University.
Beyer, H., & Holtzblatt, K. (1999). Contextual design. Interactions, 6(1), 32-42.
Retrieved 12/12/2005, from
http://www.csc.ncsu.edu/faculty/anton/csc/rep/beyer.pdf.
Boehner, K., Vertesi, J., Sengers, P., & Dourish, P. (2007). How HCI Interprets the
Probes. Paper presented at the CHI’07 - SIGCHI Conference on Human
factors in computing systems. 28 April - 3 May, 2007, San Jose, CA, 1077-
1086. Retrieved 10/07/2007, from
http://cemcom.infosci.cornell.edu/uploads/pubs/Boehner_Vertesi_Sengers_D
ourish_Probes.pdf.
Borgatti, S. P., Everett, M. G., & Freeman, L. C. (2002). Ucinet for Windows:
Software for Social Network Analysis. Harvard, MA: Analytic Technologies.
Boyatzis, R. E. (1998). Transforming Qualitative Information: Thematic Analysis and
Code Development. Thousand Oaks, CA: Sage Publications.
Braun, V., & Clarke, V. (2006). Using Thematic Analysis in Psychology. Qualitative
Research in Psychology, 3, 77-101.
Breiger, R. L., Boorman, S. A., & Arabie, P. (1975). An algorithm for clusteirng
relational data with applications to social network analysis and
comparison with multidimensional scaling. Journal of Mathematical
Psychology, 12, 328-382.
Breiger, R. L. (2004). The Analysis of Social Networks. In M. Hardy & A. Bryman
(Eds.), Handbook of Data Analysis (pp. 505-526). London: Sage Publications.
Brook, C., & Oliver, R. (2006). Exploring the influence of instructor actions on
community development in online settings. In N. Lambropoulos & P. Zaphiris
(Eds.), User-Centered Design of Online Learning Communities (pp. 341-364).
Hershey: Idea Group.
Button, G. (2000). The ethnographic tradition and design, Design Studies, 21, 319-
332.
Cerulo, K. A. (1997). Reframing Social Concepts for a Brave New (Virtual) World.
Sociological Inquiry, 67(1), 48-58.
Cuthell, J. P. (2005). What does it take to be active? Teacher participation in online
communities. International Journal of Web Based Communities (IJWBC),
1(3).
Daniel, B. K. M. (2007). A Bayesian Belief Network Computational Model of Social
Capital in Virtual Communities. Unpublished Research, PhD Thesis,
University of Saskatchewan, Canada. Retrieved 01/08/2007, from
http://library2.usask.ca/theses/available/etd-07132007-
141903/unrestricted/ben_m.pdf.
Dourish, P. (2006). Implications for Design. Paper presented at the CHI'06 - ACM
Conference on Human Factors in Computing Systems: Interact. Inform.
Inspire. 22-27 April 2006, Montreal, Canada, 541-550. Retrieved 26/01/2007,
from http://www.ics.uci.edu/~jpd/publications/2006/implications-chi2006.pdf.
Dourish, P., & Button, G. (1998). On "Technomethodology": Foundational
Relationships between Ethnomethodology and System Design. Human-
Computer Interaction, 13(4), 395-432.
Everett, M. (2005). Extending Centrality. In P. J. Carrington, J. Scott & S.
Wasserman (Eds.), Models and Methods in Social Network Analysis (pp. 57-
76). New York, NY: Cambridge University Press.
Fahy, P., & Ally, M. (2005). Student Learning Style and Asynchronous Computer-
Mediated Conferencing (CMC) Interaction. American Journal of Distance
Education, 19(1), 5-22. Retrieved 12/12/2005,
http://www.leaonline.com/doi/abs/10.1207/s15389286ajde1901_2.
Fahy, P. J., Crawford, G., & Ally, M. (2001). Patterns of Interaction in a Computer
Conference Transcript. International Review of Research in Open and
Distance Learning, 2(1). Retrieved 12/12/2005, from
http://www.irrodl.org/content/v2.1/fahy.html.
Faulkner, X. (2000). Usability Engineering. New York, NY: Palgrave, MacMillan.
Freeman, L. C. (1979). Centrality in Social Networks: Conceptual Clarification. Social
Networks, 1, 215-239.
Furniss, D. (2004). Codifying Distributed Cognition: A Case Study of Emergency
Medical Dispatch. MSc Dissertation at UCL Interaction Centre, London.
Retrieved 10/05/2006, from
http://www.uclic.ucl.ac.uk/people/d.furniss/Documents/DomProjectFinal.pdf.
Grossen, M., Pochin, L. O., Resnick, L. B., Säljö, R., Pontecorvo, C., & Burge, B.
(2006). Interactional perspectives on the use of the computer and on the
technological development of a new tool : The case of word processing.
NATO ASI series. Series F : computer and system sciences (NATO ASI ser.,
Ser. F : comput. syst. sci.) ISSN 0258-1248 CODEN NASFEG North
Atlantic Treaty Organization Advanced Science Institutes series.
Gulati, S. (2006). Learning during online and blended courses: Unpublished
Research, PhD Thesis at the Department of Education and Lifelong Learning,
City University, London.
Gunawardena, C., Lowe, C., & Anderson, T. (1997). Analysis of a Global Online
Debate and the Development of an Interaction Analysis Model for Examining
Social Construction of Knowledge in Computer Conferencing. Journal of
Educational Computing Research, 17(4), 397-431.
Guribye, F. (2005). Infrastructures for learning: Ethnographic inquiries into the social
and technical conditions of education and training. Unpublished Research,
PhD Thesis at the Department of Information Science and Media Studies,
University of Bergen, Norway.
Guribye, F., & Wasson, B. (2002). The Ethnography of Distributed Collaborative
Learning. Paper presented at the CSCL'02 - Computer-Supported
Collaborative Learning. 7-11 January, Boulder, CO. retrieved 09/05/2006,
http://newmedia.colorado.edu/cscl/148.pdf.
Hanneman, R. A., & Riddle, M. (2005). Introduction to social network
methods. Riverside, CA: University of California, Riverside.
Henri, F. (1992). Computer Conferencing and Content Analysis. In A. R. Kaye (Ed.),
Collaborative learning through computer conferencing: The Najaden Papers
(pp. 117-136). Berlin: Springer-Verlag.
Mason, R. D. (1999). IET’s Masters in Open and Distance Education: What have we
learned? CITE Report No 248. Institute of Educational Technology (IET), The
Open University, Milton Keynes, UK: The Open University. Retrieved
15/12/2005, from http://iet.open.ac.uk/pp/r.d.mason/downloads/maeval.pdf.
Massey, A. (1999). Methodological Triangulation, Or How To Get Lost Without Being
Found Out. Studies in Educational Ethnography, 2, Explorations in
Methodology, Stamford, JAI Press, 183-197. Retrieved 01/06/2006, from
http://www.voicewisdom.co.uk/htm/triangulation1.htm.
Meyer, K. A. (2002). Quality in distance education: Focus on on-line learning. San
Francisco, CA: Jossey-Bass.
McGarry, B. (2005). Things to Think With: Understanding Interactions with Artefacts
in Engineering Design. Unpublished Research, PhD thesis at the Department
of ITEE, School of Information Technology and Electrical Engineering,
University of Queensland. Retrieved 14/05/2006, from
http://eprint.uq.edu.au/archive/00003422/01/McGarry%20PhD%20Thesis%20
-%20Things%20to%20Think%20With.pdf.
Milic-Frayling, N. (2007, 1 May 2007). Internet - Where Information and
Communication Technologies Meet. Paper presented at the Women in
Computing Research: the Hopper Colloquium. 1 May 2007 London Hopper
2007, British Computer Society, London.
Moodle@GSN (n.d.). Moodle at the Greek School Network, http://e-learning.sch.gr.
Nielsen, J. (1993). Usability Engineering. Boston, USA: Academic Press.
Nonnecke, B. (2000). Lurking in email-based discussion lists. Unpublished Research,
PhD Thesis at London South Bank University, London.
Nooy, W. d., Mrvar, A., & Batagelj, V. (2005). Exploratory social network analysis
with Pajek. Cambridge: Cambridge University Press.
Paiva, V. L. M. d. O. e. (2005). A pesquisa sobre interação e aprendizagem de
línguas mediadas pelo computador (A research on computer-mediated
interactions in language learning). Calidoscópio, São Leopoldo, 3(1), 5-12.
(Translated by Margarida Romero.) Retrieved 22/05/2006, from
http://www.veramenezes.com/cmc.htm.
Petropoulou, O., Lazakidou, G., & Retalis, S. (2006). Πώς να Αξιολογήσουμε τη
Διαδραστικότητα σ’ ένα Τεχνολογικά Υποστηριζόμενο Περιβάλλον Μάθησης;
How can we evaluate interactivity in a technology enhanced learning
environemnt? Paper presented at the 3ο Πανελλήνιο Συνέδριο Ε.Ε.Ε.Π. -
Δ.Τ.Π.Ε. "Εκπαίδευση & Νέες Τεχνολογίες". 3rd Panhellenic Conference
EEEP-DTPE "Education and the New Technologies", 23-46. 30 September -
1 October 2006, Korydallos, Athens, Greece.
Preece, J., Rogers, Y., & Sharp, H. (2002). Interaction Design: Beyond Human-
Computer Interaction. New York, NY: John Wiley & Sons, Inc.
Preece, J. (2000). Online Communities: Designing Usability, Supporting Sociability.
Chichester, UK: John Wiley & Sons.
Rice, R. E., & Love, G. (1987). Electronic emotion: Socio-emotional content in a
computer-mediated communication network. Communication Research,
14(1), 85-108.
Robinson, H., Segal, J., & Sharp, H. (2007). Ethnographically-informed Empirical
Studies of Software Practice. Information and Software Technology. In press.
Sanson-Fisher, R. (2004). Multiple Baseline Designs. Paper presented at the
Workshop on Research Designs for Complex, Multi-level Health Interventions
and Programs. 4 -5 May, 2004, Bethesda, MD. Retrieved 11/01/05, from
http://obssr.od.nih.gov/Conf_Wkshp/Complex%20Interventions%20Workshop
/Sanson-Fisher.ppt.
Sasse, A. (1997). Eliciting and Describing Users’ Models of Computer Systems.
Unpublished Research, PhD Thesis, University College London, London, UK.
Retrieved 15/02/2006, from
http://www.cs.ucl.ac.uk/staff/a.sasse/thesis/Frontpage.html.
Shadish, W., Cook, T., & Campbell, D. (2002). Experimental. & Quasi-Experimental
Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin.
Sharp, H., Robinson, H., Segal, J., & Furniss, D. (2006). The Role of Story Cards and
the Wall in XP teams: a distributed cognition perspective. Paper presented at
the Agile 2006. 23-28 July 2006, Minneapolis, Minnesota, USA. Retrieved
12/05/2006, from
http://www.dcs.qmul.ac.uk/research/imc/hum/pubs/SharpEtAl-
CardsAndWallFinalVersion1%20PREPRINT.pdf.
Sharp, H., & Robinson, H. (2005). Some social factors of software engineering: the
maverick, community and technical practices. Paper presented at the
HSSE'05 - 2005 Workshop on Human and Social factors of Software
Engineering. 16 May 2005, St. Louis, Missouri. Retrieved 12/05/2006, from
http://delivery.acm.org/10.1145/1090000/1083117/p11-
sharp.pdf?key1=1083117&key2=8932278711&coll=&dl=&CFID=15151515&C
FTOKEN=6184618.
Shneiderman, B. (2000). Creating Creativity: User Interfaces for Supporting
Innovation. ACM Transactions on Computer-Human Interaction, 7(1), 114–
138.
Silius, K., & Tervakari, A.-M. (2003, 8-9 May 2003). An Evaluation of the Usefulness
of Web-based Learning Environments.The Evaluation Tool into the Portal of
Finnish Virtual University. Paper presented at the International Conference on
Network Universities and E-Learning, Valencia, Spain. Retrieved 23/04/2006,
from http://matriisi.ee.tut.fi/arvo/liitteet/usefulness_of_web.pdf.
Suchman, L. A. (1987). Plans and Situated Action: The Problem of Human-Machine
Communication New York: Cambridge University Press.
Suter, E. A. (2000). Focus Groups in Ethnography of Communication: Expanding
Topics of Inquiry Beyond Participant Observation The Qualitative Report, 5(1
& 2).
Talamo, A. (2005). Developing, Animating and Analysing Virtual Communities of
Practices. Paper presented at the Workshop on Learning Communities in the
era of Ubiquitous Computing. 13 June 2005, Milan, Italy. Retrieved
21/05/2006, from
http://www.idi.ntnu.no/~divitini/ubilearn2005/Final/talamo_ubilearn.pdf.
Talamo, A., & Ligorio, M. B. (2002). Ethnomethods as resources for developing
CVEs in the ITCOLE project. Paper presented at the CSCL'02 Conference. 7-
11 January, Boulder, Colorado, USA. Retrieved 21/12/2005, from
http://newmedia.colorado.edu/cscl/167.htm.
Tedlock, B. (2000). Ethnography and ethnographic representation. In N. K. Denzin &
Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed) (pp. 455-486).
Thousand Oaks, NJ: Sage.
Walther, J. B. (1996). Computer-mediated communication: Impersonal, Interpersonal
and Hyperpersonal Interaction. Communication Research, 23, 3-43.
Wellman, B. (2001). Physical Place and Cyber-Place: Changing Portals and the Rise
of Networked Individualism. International Journal for Urban and Regional
Research, 25(2), 227-252.
White, H. C., Boorman, S. A., & Breiger, R. L. (1976). Social structure from
multiple networks: I. Blockmodels of roles and positions. American
Journal of Sociology, 87, 517-547.
Zaharias, P. (2004). A Usability Evaluation Method for E-Learning Courses.
Unpublished Doctoral Dissertation at Athens University of Economics and
Business.
Chapter 4 reveals aspects of the research context with the aid of ethnotechnology.
This is connected to the Greek educational authorities’ intentions, targets, and tools
provided for the Greek teachers’ professional training and development. One of the
tools was Moodle@GSN, the e-learning platform supported by the Greek School
Network. The chapter also sheds light to the Greek teachers’ needs, background,
and characteristics as well as their overall activity at Moodle@GSN.
4.1 INTRODUCTION
Because some parts of the jigsaw were not found in Greece, observant participation
in other groups was considered necessary. Thus the following projects were also
used to provide information:
Access was provided by the researcher’s colleagues and there were discussions with
the organisations’ directors as well as members.
widely known for members’ genuine interest and creative inputs in the field of online
communities. Among other results, thematic analysis revealed a template for writing
online messages, and a community management framework (Appendix I).
The results revealed that the Greek teachers had difficulties in integrating new
practices learned from online seminars in their teaching and learning practice. The
main reasons were the lack of training, lack of opportunities, lack of soft skills, and
absence of professional guidance, help and support.
Lastly, I acquired the opinions of the ‘Greek Primary Teachers’ Association for the
Valorization of ICT in Education’ (EEEP, http://www.eeep.gr; formed in December
2003). Fourteen out of 61 participants (22,95%) responded to a questionnaire in July
and August 2004. The factors that appeared to influence members’ participation,
were: (a) organisational, related to educational authorities; (b) school-based; (c)
personal: age, gender, training or absence of training on the use of computers, years
of teaching, previous experience and familiarization with ICT, writing and typing skills,
and personal characteristics; and (d) “the real world”: for example, expensive rates
for the internet access in Greece, the Internet connection from the schools,
participants’ spare time and time available for training. (The results of these studies
have been published in Lambropoulos 2005a, 2005b, 2005c.)
These stages appear to be important; for example, initiations and interventions need
to be applied after the first week in order to allow time for the e-learners to familiarise
with the system and the community. The next section investigates the context of
educational authorities and GSN.
The Moodle@GSN teachers are located within the black arrow area; they teach
different subjects and have different roles in the Greek education system. GSN asked
e-tutors and teachers to participate on a voluntary basis.
• teachers’ training needs at school level were concerned to the social relations
and collaborative activities within the school environment;
• teachers needed support and training in collaboration, practical application of
innovations as part of scientific and professional training;
• there was lack of self-esteem and self-confidence derived from insufficient
training (Tsetsilas refers to “insufficient cognitive equipment”, 2006:6);
• the basic characteristics of the Greek educational system and the official
educational policy was centralism, restrictions in school autonomy, insufficient
basic and ineffective professional training; and
• certain parameters in the school environment required training.
the Greek teachers’ demonstrations continued every Wednesday until this section
was written (18/02/2007). Therefore a risk analysis was needed (Appendix II).
Overall, it appears that there were discrepancies between the stakeholders’ goals,
common ground and right attitude are not established. However, there is space for
improvement on a social, cognitive, and technical level.
4.3 PLANNING
Ethnotechnology can capture relevant information for planning in e-learning
engineering. The first section described the Greek teachers’ context; this section will
refer to the e-learning environment.
The Greek School Network (GSN) is the educational intranet similar to the
British JANET (Vivitsou et al., 2008). With regards to the e-learning service,
evaluation of various open-source learning management systems led its developers
to the open source package Moodle (Modular Object Oriented Dynamic Learning
Environment; http://www.moodle.org). The GSN e-learning platform is called
Moodle@GSN (http://e-learning.sch.gr). The Greek teachers played the roles of both
the course creators and students on a voluntary basis. Moodle@GSN is an
autonomous, self-organised service for teachers’ distance education training (Figure
4.3.1-1):
Moodle@GSN
Moodle@GSN started in 2003 with 2 online courses. Nine more courses were
created in 2004, 2005 and 26 in 2006, in a total number of 46 courses. The baseline
was the 1st of November 2006; on this date, there were 1,910 users registered online
whereas 64 users were registered in the 14 courses that were under construction. In
a total number of 4,350 members, there are 853 registered active participants, and
25 e-tutors. This means that 19% of the total number of registered users exhibited
some kind of activity (e.g. visiting the online courses, posting in chats and
discussions) in Moodle@GSN and 3% were the e-tutors. Due to the fact that
participation was on a voluntary basis, the offered courses had an open nature and
were divided in two major categories, the structured and well managed and the
courses that were designed for a special interest (e.g. Yoga, bird watching). With the
subject as the criterion, the categorization of the online course (Table 4.3.1.1-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 100
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context
General Interest
1400
1200 Greek Schools Netw ork
1000 Training
Learners
600
Pedagogical Approaches
400
200 IT Training
0
1 2 3 4 Multimedia
Category
Open Office
It appears that in 46 completed and courses under construction there were 1,910
Greek teachers having a preference in IT training, multimedia, as well as the use of
pedagogical approaches. From the overall 206 messages sent in these courses, 8
discussion threads started from the e-tutors and 18 from the e-learners themselves,
resulting in 132 messages from the e-tutors and 74 from the e-learners There were
94 new topics of discussion launched by 34 e-tutors. In 22 courses there were chats,
having 1,000 messages sent by 138 participants. In addition, 40 quizzes were
activated in 19 courses used by 106 users in 197 efforts to solve them. Lastly, there
were 67 assignments in 18 courses from 47 users in a total of 94 submissions.
The 1st of November 2006 (baseline) was the starting point of interventions. There
were two interventions, collaborative e-learning and the use of tools based on the
proposed underlying conceptual frameworks. Part of the collaborative e-learning
intervention was the introduction of community management by the investigator. It
consisted of material providing advice on how to better use the platform including
emoticons. It also had material intended to improve the participants’ soft skills for
effective communication and about collaborative e-learning principles (Appendix VI).
The second intervention was to provide access to the new tools.The following table
describes the type and the number of messages. The ratio of single messages
26.6%, in a total number of 206 messages, 16.5% were the messages with a reply,
and 56.7% were the overall replies. In addition, it appears that there is counter
analogy to the number of messages and the number of replies, for example 19
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 101
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context
messages had 1 one reply, whereas 1 message produces 27 replies (Graph 4.3.1.1-
2):
30
25
20
Number of Messages
15
Number of replies
10
0
1 2 3 4 5 6 7 8 9 10
This message was initiated to test the e-mint community managers’ suggestions;
they had to talk about their experiences and the Greek teachers responded
enthusiastically. The intervention stopped in order not to interfere with the study.
From the total number of 32 courses, two courses produced the aforementioned
messages. One course was built before the intervention, and the second after.
The messages in the first course ‘Use of ICT in Religious Education (RE)’ on the
discussion topic about the use of Power Point presentation were limited as regards
the richness of the text. The messages were sent during the period 26/03/2006 -
29/03/2006. There were 28 words in 6 threads. Even though the discussion is limited
there is evidence of collaborative learning. The participants (all different individuals)
built on each others’ threads and agreed on the importance of collaboration as
follows:
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 102
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context
Next, the participants were invited to the second course on project management, built
from the researcher with the help of two colleagues, Marianna Vivitsou and Dimitris
Konetas. The participants were given educational material (samples exist in
Appendix VI) and the e-tutors followed the online community management framework
(Appendix I). The course “Project Method” was designed to respond to the Greek
educators’ need for training on educational project management, institutionalised by
law (Greek Government Gazette 303 & 304/13-3-2003).
The number of visits and the number of messages sent were (Table 4.3.1.1-2):
It appears that 4,576 users posted 34 messages with 708 views. The course under
construction for this study appeared to have initiated a significant number of posts. A
comparison demonstrates the rapid increase of the messages in the course “Project
Method” after the initiation of the community management scheme (Table 4.3.1.1-3):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 103
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context
The e-learners, all different individuals, produced 14 threads with more than 2,000
words in total with the Project Method course among them with 30 replies between
04/06/2006 and 01/11/2006 presenting the following activity in the first week (Graph
4.3.1.1-3):
13, 10% 8, 6%
Number of Topics
28, 22%
Number of new
discussions
Number of sent
messages
Number of active
participants
77, 62%
It appears that the there was initial significant activity after the baseline and the first
intervention; there were 8 topics, 28 new discussions, 77 sent messages and 13
active participants. From the total number of 45 participants, 71% were passive and
29% active participants with an average of 6 messages per poster. Overall, it
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 104
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context
The interventions in this research were the support of collaborative e-learning and its
social aspects in particular as well as the introduction of a set of new tools to support
passive and active participation levels, interactions, and the collaborative e-learning
as such. Thus, the studies were as follows (Table 4.3.1.1-4):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 105
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 106
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 4: Research Context
REFERENCES
Dourish, P., & Anderson, K. (2006). Collective Information Practice: Exploring Privacy
and Security as Social and Cultural Phenomena. Human-Computer
Interaction, 21(3), 319-342.
Fernandes, C., & Montalvo, A. (2006). E-Quality Synthesis Report: Experience-based
Quality in European ODL.
Golafshani, N. (2003). Understanding Reliability and Validity in Qualitative Research.
The Qualitative Report, 8(4), 597-607.
Lambropoulos, N. (2005). EEEP Online Community of Practice: The First to Boldly
Go in Greece. "Open Education", The Journal for Open and Distance
Education and Educational Technology, 2, 29-56.
Lambropoulos, N. (2005). Online Empathy: Social Consequences for Community
Building. In S. Dasgupta (Ed.), Encyclopaedia of Virtual Communities and
Technologies (pp. 346-348). Hershey, PA: Idea Publishing.
Lambropoulos, N. (2005). Paradise lost? Primary Empathy in Online Communities of
Interest and Ways of Use. Paper presented at the 1st Conference on Online
Communities and Social Computing, in the 11th International Conference on
Human-Computer Interaction 2005, 22-27 July, Las Vegas, NV. Retrieved
05/09/2005, from
http://www.intelligenesis.eu/nikiweb/PDF/05/lambropoulosOnlineEmpathyVeg
as05Research.pdf.
Shackel, B. (1991). Usability – Context, framework, definition, design and evaluation.
In B. Shackel & S. J. Richardson (Eds.), Human Factors for Informatics
Usability (pp. 21–37). New York, NY: Cambridge University Press.
Tsetsilas, I. (2006). Reporting teachers' training needs and design training schemes:
The case study from a primary school using D.I.O.N. model. Καταγραφή
επιµορφωτικών αναγκών των εκπαιδευτικών και σχεδιασµός προγραµµάτων
επιµόρφωσης: Η µελέτη περίπτωσης µίας σχολικής µονάδας της
Πρωτοβάθµιας Εκπαίδευσης µε το µοντέλο D.I.O.N. . Unpublished research at
the Open University, Patras, Greece.
Vivitsou, M., Lambropoulos, N., Konetas, D., Paraskevas, M., & Grigoropoulos, E.
(2008). The Project Method e-course: the use of tools towards the evolution
of the Greek teachers’ online community. International Journal of Continuing
Engineering Education and Lifelong Learning (IJCEELL), 18(1), 26-39.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 107
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
Chapter 5 presents the initial design and evaluation related to the conceptual
frameworks, the tools prototypes, the evaluation techniques, and the research
design. Three studies are presented, one with international participation, one with the
Greek teachers, and one focus group with Greek teachers who are also Moodle
developers. Guidelines and heuristics obtained from feedback on the proposed
concepts and tools.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 108
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
5.1 INTRODUCTION
Six tools 1 were initially considered for integration in Moodle, and 3 tools were
found which were either not working for the Greek context or not suitable to be
discussed here; these tools were:
1
The tools were developed by Intelligenesis, a British company specialising in market research, social
networking and e-learning.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 109
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
This section will present the iterative design process from creating the first
designs to the actual re-design and implementation. Four projects (blocks) were
developed, two of them will be discussed here (Table 5.2.-1):
ITERATIVE DESIGN
Blocks Sample Environment Intervention Activity
Greek
-Learning Online course:
21/03/2006 education Moodle@GSN
1 Management Introduction to Web
-07/04/2006 mailing lists, & E-mmersion
Tools Design
EFQUEL list
Online course:
01/11/2006 Learning Educational Project
2 Moodle@GSN Moodle@GSN
-30/11/2006 Management Management with
Collaboration Tools
Greek Moodle
06/03/2007 Recommendations
3 developers E-mmersion -Tools
-13/03/2007 for new tools
lists
Online course:
-Learning
01/03/2007 PSD e-mail Moodle@GSN Educational Project
4 Management
-31/03/2007 database & E-mmersion Management with
-Tools
Collaboration Tools
Block 1 and 3 will be discussed in this chapter. Block 2 tested the community
management as part of collaborative learning framework, and will only be used for
comparison. Block 4 will be discussed in the next chapter as the main study.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 110
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
The literature revealed that observing and analysing social interactions can
increase collaborative learning understanding. JUNG Social Network Analysis (SNA)
open source software was found to be suitable to acquire real-time results. Social
interactions can be depicted in SNA Nodes and Centrality windows as Visualisation
Interactions tools (VIT) open as Java applets using algorithms (Figure 5.2.1.1-1):
(a) (b)
Figure 5.2.1.1-1. Visualisation Interaction Tools (VIT) Nodes and Centrality
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 111
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
Moodle was patched to create a link which passes the forum ID to a Java applet. The
applet executed PHP script that fetched all required information from the Moodle
database creating an XML file, with all calculations made such as user relations and
message count. Then XML passed to the JAVA applet and the JAVA applet built a
graph based on information generated from the XML file.
The hosting company upgraded PHP4 to PHP5 and constantly reconfigured PHP5 and
Apache; JPGraph, the graphing software creator, was not updated, whereas GD library
was not activated. GD creates PNG, JPEG and GIF images and is commonly used to
generate charts on the fly. Not being activated after the upgrade resulted in major
problems with the use of these tools; the images did not display correctly and were
represented by an X. Participant D1 expressed his frustration: ‘It keeps driving me
bananas all the time. It really gives me the willies when that peculiar X appears in the top
lefthand corner’.
Despite the problems, e-tutors and e-learners would be able to observe the
sociability of the human network.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 112
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
Moodle was patched in order to auto-generate links with the course ID that pointed to
PHP scripts in order to enquire users’ database accessing the specified course. Then
it gathered all statistical information from databases, parsed information and passed
it to JPGraph library in order to perform an output on the screen in real-time.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 113
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
The script for such a tool is very simple, written in PHP and HTML and can be
incorporated in any Post page. Moodle was patched to allow all messages stored in
the database to be tagged. In addition to the tool, SQL queries could be performed to
get different statistical information based on message types or complex statistics
where types are only a part of a query (for example keyword and message type that
was already implemented but not used). The use of icons was considered instead of
text as used in the “vicarious learner” project (Lee et al., 1999). However, it was
thought that there will be excessive iconic information on the interface and, since the
e-learners actually read the title and the actual message when they post, the use of
text would not interrupt their reading but can be part of it.
Overall, it is anticipated that the tools can broaden and illuminate the space between
human-human interactions so as to open space for reflection for the e-learners. From
a research viewpoint, the tools could expand our understanding on the socio-
cognitive aspects of collaborative e-learning.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 114
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
The call for participation was sent initially to registered users in the web design
course at Moodle@GSN; only 5 out of 128 registered users responded to the call
(28/01/2006). Help was sought and found from the European Foundation for Quality
in eLearning (EFQUEL) (http://www.qualityfoundation.org/). The call was also sent to
the most popular Greek educational mailing lists, EEEP and PEKADE as well as
EFQUEL, and was announced on EFQUEL front page.
The online course in the experimental environment was supplementary to the web
design course in Moodle@GSN. From the 68 initial subscribers 12 did not enrol. The
remaining 56 participants are shown in the following graph (Graph 5.2.2-1):
Participating Countries
1, 1, 1, 1,
2%2% 2%2%
1,1,2%
2%
Greece
1,
1, 1%2%
2, 2% UK
Hungary
Germany
India
13, 19% Portugal
Spain
Sweden
43, 64%
Switzerland
USA
Turkey
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 115
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
The majority of the participants were from Greece (n=43, 64%) and the UK (n=13,
19%). The participants were divided in two groups and worked in two research pools,
the web design course for the Greeks and a discussion to evaluate the tools for all.
Also, there was a ‘Living Room’ for all participants. The participants were free to
communicate in both languages, English and Greek, however, English was proposed
in the tools evaluation discussion.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 116
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 117
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
Figure 5.2.2-2 shows the code network for the 4 CeLEs in the Web Design pool. The
codes were: initiation, explanation, agreement, conflict, exploration, evaluation, new
ideas for co-construction, and other. After the analysis, it appeared that social cues
can be a separate attribute for the CeLE framework. Additionally, disagreement was
collapsed with agreement as they are opposite without being separate.
The next section presents the results from the second pool relevant to this chapter,
the tools evaluation pool.
In the tools evaluation pool there were 7 low, 2 medium and 1 high activity
user and a total of 33 messages were sent. (The high activity user A1 was the
Swedish Moodle developer.) (Figure 5.2.2.1-1):
In addition to the pattern for writing online messages from the E-mint study, a second
information about the locus of participants within one discussion in a forum, based on
Social Network Analysis and the levels of participation. The two tools were located on
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 118
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
the top of discussion (Figure 5.2.2.1-2) and were VIT Nodes (Figure 5.2.2.1-3) and
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 119
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
A1 suggested that learners ‘can lurk intensively and actively and learn more than
those visibly active’, so she proposed passive participation to be visible on VIT. The
participants had to face new concepts and work with new tools without previous
experience. Participant M1 suggested: ‘…i (at least tried to) familiarised with new
(once again) that there are some many things left for me to ‘learn’ in the LMS
concepts need to be defined (e.g. what are ‘best learning achievements?”), some
applications / tools to be improved (e.g. philippa’s repertoire of responses), some to
be further explored (e.g. VIT tools, activity graphs, message tags) and the moodle
Participation Evaluation Graphs: Two types of graphs were created associated with
the online course and the individual e-learner describing the e-learning community
progress (Graphs 5.2.2.1-1 to 5.2.2.1-3):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 120
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
As viewed in the previous graphs, only two members appeared in high and medium
activity during the first days (31/03/2006). Thus the moderator sent an email
suggesting that all participants need to contribute to the discussion. This intervention
appeared successful; 5 more participants appeared in the low participation graph
5.2.2.1-2 (04/04/2006).
M1 said that such a tool is not helpful: ‘for a newbie like me passive participation very
often proves to be very constructive’ (M1). E1 found difficulty in interpreting them. A1
said: ‘they show that e.g. my activity level in the Induction course is medium, that
there are 5 other users on low level and 1 user, probably admin with high level.’ To
A1 the kind and the quality of the activity is missing ‘or whether anything, and if so
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 121
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
what, may have been learnt.’, so log files provide more valuable information. E1 said:
‘they are more useful to teachers than fellow students.’. M1, talking about the sense
of community, said: ‘how can you, from in this way graphically presented statistics,
can conclude that someone is 'immersed'.’.
From an e-tutor’s and researcher’s viewpoint, the graphs were effective. Similar tools
exist in other learning management systems such as Blackboard, where bar graphs
depict e-learners’ overall activity rather than distinguishing passive and active
participation levels. There were some issues due to systems’ instability; the first
column of participation was unstable; passive participants were not shown on the
graphs (participation column should have shown 28 participants in Graphs 6.2.2.1-1-
3). In addition, it was not possible to retrieve their log files. Other problems were
related to more than one login names registered for each user exhibiting multiple
appearances, indicating the need for an initial account confirmation to ensure
accurate and reliable results.
The tool was used more in the Evaluation Pool than the Web Design pool as follows
(Table 5.2.2.1-1) (only the participants’ messages were counted):
2
At this point in the study, disagreement was coded under evaluation. Later it appeared as on option in
conjunction with agreement
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 122
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
USE OF MESSAGETAG
Overall CeLE Tagged
Pools # Percentage
replies Attributes messages
Inform 9
Living Question 2
Room 35 17/35 48.5
Explore 4
Explain 2
Inform 4
Web Question 8
Design 73 Explore 5 20/73 27.3
Explain
3
Inform 12
Question 6
Tools
47 Explore 2 26/47 55.3
Evaluation
Explain 4
Evaluate 2
Inform 25 39.6
Question 16 25.3
Total 155 Explore 11 63/155 17.4 40.6
Explain 7 11.1
Evaluate 2 3.1
It appears that MessageTag was mostly used in the tools evaluation pool with 26 out
of 47 messages tagged (55.3%) and less in the Greek teachers’ only pool with 20 out
of 73 messages tagged (27.3%). In the introductory forum (Living Room) 17 out of 35
messages were tagged (48.5%). Because the tool was integrated in discussion, the
log files are the same with the forum and discussion logs. Based on the tagged
messages, the inform tag was used more than the other tags (39.6%) and the
evaluate tag less (3.1%); 16 (25.3) messages were tagged as questions, 11 (17.4) as
explorations, 7 (11.1%) as explanations, and 2 (3.1) as evaluations.
A1 found this tool interesting whereas E1 said that she never saw a tool like this
before so she did not know the purpose it served or how it can be used (‘for statistics
maybe?’). She wondered whether ‘users read messages according to their tagging
(from a long list of messages). A help file next to "message tag" would be useful.
Perhaps the text of each tag could be accompanied by an icon.’. AM and M1 agreed
with E1. M1 said that ‘each tag provides a thematic categorisation of the reply.
personally i read all the replies to the topic i'm interested in no matter what the tag is,
perhaps because what matters is the content and not the title. it might prove more
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 123
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
useful for younger learners' induction to the system though’. G1 agreed with M1. A1
added that the tool would have worked better if it could show the very structure of the
discussion as not many people are capable ‘of seeing such structures by themselves,
this meaning that they are likely to miss, or misunderstand the crucial points. All
learners are not good at 'deep' learning.’. He suggested that this tool could be
involved into becoming an instrument ‘for learners to enhance their literacy i.e.
becoming better e-readers/listeners and e-writers/speakers, then that would certainly
be a 'huge step for humanity'’. A1 recommended some additional tags for
consideration:
• add/develop
• ask for additional information
• ask for clarifications
• confirm
• approve
• agree
• disagree
• offer conclusions
• point to resources
Lastly, it was evident that the use of MessageTag attributes was descending as the
level of reflection and higher order thinking was ascending. This finding stresses the
importance attached to the difference between information provision and knowledge
acquisition and that the tool can make this difference visible. In this way the e-
learners participants can moderate their own messages and use the tool to facilitate
their critical thinking skills development; as such, the e-tutors can support this
process. This will have an impact on the e-learning quality.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 124
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
Participation Evaluation Tools: The two graphs for overall course participation
levels and discussion participation levels had to depict all participation levels for all
participants so there were three changes (Figure 5.2.2.2-2):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 125
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
The avatars were integrated on the threaded discussion view. This means that the
participants can compare collaborative e-learning structures and participation levels
in one glance. Lastly, the log files will be the same as the discussion forum log files.
MessageTag: First, two more values were added on MessageTag, Agree and
Summarise. A1 suggested adding conflict but the decision was made; conflict was a
subcategory used for coding only along with disagreement. So the new collaborative
e-learning episode analytical framework was redesigned (Figure 5.2.2.2-3):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 126
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
1. Questionnaire: The e-learners did not use the real-time online questionnaire
(N=9/31, 29%); several changes were made so the questionnaire could be
sent via email:
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 127
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
Thus, the research design was adapted as follows (adaptations in grey) (Table
5.2.2.3-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 128
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
ETHNOTECHNOLOGY DESCRIPTION
1. Participation in projects, conferences, seminars,
online discussions of Greek teachers’ associations
and schools in Greece
1 Scenic Fieldwork 2. Documentation, manuals and presentations by the
Greek educational authorities
3. Personal opinions, conversations, and emails
4. Richness of text
1. Demographics
2 Pre-Questionnaire 2. Conditions of working and learning over the Internet
3. Initial knowledge on collaboration
1. Frequency of visits
2. Number of active and passive participants
3. Active participation levels
4. Passive participation levels
5. Sense of e-Learning Community Index
3 Logging • Persistence: depth of discussion threads
• Density: formula 2aN(N-1)
• Reciprocity: number of messages sent and
received
• Intensity: levels of participation
1. Collaborative e-learning episodes analysis
4 Thematic Analysis 2. Sense of e-Learning Community Index
• Number of Collaborative e-learning episodes
The Sense of e-Learning Community Index
1. Global Cohesion
• Density
• Reciprocity
Social Network • Cliques
5 Analysis • Structural equivalence
2. Global Centrality
• Centrality
• Closeness
• Betweenness
3. Local real-time nodes & centrality
1. The Sense of e-Learning Community Index
• Community evolution
• Sense of belonging to the e-learning community
• Empathy
6 Post-Questionnaire • Trust: knowledge exchange, help and support
• Collaborative e-learning quality: participants’
opinions
2. Usage and usability of collaborative tools
3. Professional development
1. Examination of the data, missing data, normality.
2. Descriptive Statistics: crosstabulation, frequencies,
arithmetic means, standard deviations, and
7 Statistical Analysis exploration
3. Correlation: Pearson correlation coefficient (r).
4. Inferential statistics: statistical significance, p-value
(Cronbach's alpha, α), and null hypothesis.
1. Form of the effect: the level, slop, variance and
cyclicity
8 Intervention Analysis
2. Its permanence (continuous or discontinuous)
3. Its immediacy (immediate or delayed)
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 129
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
Overall, PHP applications cannot easily work with algorithms and Java. This perhaps
indicates the need for new programming techniques to support systems
interoperability. Since such programming has not been used yet in available e-
learning management systems a new hosting company was needed to meet most of
the requirements. Furthermore, if such tools and evaluation techniques do not exist, it
is difficult for the participants to observe they are missing.
With the provided time and resources in this study only redesign of the tools was
feasible. The redesign was tested and developed further with the help of a focus
group of Greek teachers Moodle developers.
Table 5.2.3-1. Demographics for the three Greek Teachers / Moodle Developers
Demographics
Use of
Working
computer Use of Use of
Age Gender Occupation Experience
/ software Internet Moodle
(years)
(years)
1 30-40 Male Teacher 1-5 6-10 All day 1-5 years
2 40-65 Male Instructor 11-20 6-10 All day 1-5 years
Once -
Doctor /
3 40-65 Male 11-20 1-5 twice a 1-5 years
instructor
day
All three were male, two between 40-65 years old and one between 30-40 years old
and none of them had participated in any of the previous blocks or studies. P1 was a
teacher, P2 an instructor, and P3 a doctor, all working, training adults using Moodle
for 1-5 years. P1 had 1-5 years of professional experience, he was using educational
software for 6-10 years, and he had been connected on the internet all day. P2 and
P3 had the same working experience, 11-20 years; P2 was using the Internet all day
and P3 once or twice a day. None of them said he knew any tools and evaluation
techniques to support participation whereas all of them thought participation
necessary. As for the reasons, P1 said that ‘self-efficiency is important in e-learning’,
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 130
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
The discussion took place in the experimental environment with 11 discussion topics
initiated by the researcher with a brief description of each tool. Thematic analysis
was used on the 53 replies to categorise participants’ opinions on the tools; in
addition, the new collaborative e-learning scheme was tested for use and usefulness.
The discussions topics were: an introduction, new ideas, tools, and suggestions on
Greek names. Only discussions on the new ideas and tools are presented here. The
comments about the tools on the provided questionnaire were (Table 5.3.1-2):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 131
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
P1, P2 and P3 also answered the second session of the questionnaire based on a
Likert scale 1-5 and focused on efficiency and effectiveness, satisfaction (ISO DIS
9241-11, 1994), enjoyability, learnability, and overall evaluation for imagination and
satisfaction (Zaharias, 2004; Silius et al., 2003) (Table 5.2.3-3):
PEDAGOGICAL USABILITY
1 Efficiency & Effectiveness μ = 3.9
Participation
Message Tag VIT Nodes VIT Centrality
Tools
μ 4.3 3.3 3.6 4.3
2 Satisfaction = 4
Participation
Message Tag VIT Nodes VIT Centrality
Tools
μ 4.3 3.6 3.6 4.6
3 Enjoyment = 4.5
Participation
Message Tag VIT Nodes VIT Centrality
Tools
μ 4.6 4.3 4 5
4 Learnability = 4
Participation
Message Tag VIT Nodes VIT Centrality
Tools
μ 4.3 4.3 4 3.6
5 Imaginative μ = 4.6
6 Satisfaction μ = 4.3
Overall Score μ = 4.2
The tools scored 4.2 out of 5 as regards pedagogical usability. This means that the
participants were overall satisfied with the tools but made additional
recommendations as seen previously. Lastly, there were some final propositions. P2
suggested that the interface should provide two views; an overall users/learners’ view
so the moderator can support the learners’ individuality; and a user/learner’s view for
self-organisation and self-learning. He also suggested the group format for e-learners
flexibility. In this way, a user/learner who lost his interest and lurked can be activated
in a different team. Finally, P3 indicated the problem spotted in the literature,
structuring or not structuring information: ‘I think I understood!!!no need for explaining
the use of tags. But is it necessary to make use of tags? Isn't it easy to understand
the nature of the writer's message?’
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 132
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
However, this measured was found inadequate to accurately measure utility and
usability for new tools. The Collaborative e-Learning Episodes (CeLE) was successful
and 7 CeLEs were identified.
When this discussion finished the Greek teachers were in the middle of the
online course in Moodle@GSN (main study). Meanwhile work was carried out
collaboratively with another programmer to update the tools and she actively
participated in the course as an e-tutor. The considerations on design and research
design were:
• The participants did not immediately understand the usefulness of the tools.
This may partly be because they were not familiar with such tools, and partly
because of design.
• The names of the tools came from social network analysis references; the
participants did not provide any alternative names.
• The lurkers’ login names on VIT Centrality were not removed because there
was no other way to make them visible on the tool.
• Providing all information on the interface was found to be of poor usability
because of the information overload on the interface so no additional
information was provided.
• The use of one colour for each participation level did not show the
user/learner’s potential. However, this was a good idea and the high
participation graph was changed to yellow ( ). The proposed red colour was
found inappropriate due to alert signalling of red and usability purposes.
• The icons for the VIT were changed to icons acquired from a discussion.
• Two views of the tools, as well as having the tools in a group format could not
be implemented in a study such as this one. It is a bigger project and requires
team working as well as time and funding.
• The pedagogical usability questionnaire was elaborated in more detail.
• A combination of participation levels and CeLE from a bird’s eye view can
provide comparable information (Figure 5.2.3.1-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 133
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
Lastly, since the utility and usability attributes were found inadequate, they were
transformed; new attributes were anchored in Zaharias usability questionnaire,
focused on 5 out of his 13 suggested attributes (Zaharias, 2006:198-199) as well as
Preece and Maloney-Krichmar’s framework on sociability and usability for online
learning communities (2003). Further correlation analysis suggested the
interactivity/engagement and motivation to learn to be combined under ‘motivation to
participate’. Moreover,
Considering users’ unfamiliarity with new tools the utility & usability questionnaire
items were developed as follows (Figure 5.2.3.1-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 134
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
Overall, all participants agreed that the tools, and in particular the Visualisation
Interactions Tools (VIT) were useful and important, especially for the moderator,
because she can: direct and control online discussions; discover interlocutors’
weaknesses and strengths; activate the lurkers with specific questions; aid in team
building by “bounding” the team; and record the discussion. They also said that self-
efficiency is important in e-learning and feeling part of the community is what
participation is about, addressing learning on an individual and a social level.
From a researcher’s viewpoint, the numeric assessment provided by the tools and
evaluation techniques was successful. Despite the discrepancies with the graphics
and difficulty in understanding the meaning of the tools, the Greek teachers Moodle
developers were satisfied with the tools, they enjoyed using them, and asked for
more.
The last part of this research is the main study discussed in the next chapter.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 135
Tools & Evaluation Techniques for Collaborative E-Learning Communities Ch. 5: Tools & Evaluation Techniques
REFERENCES
Alty, J. L. (1997). Multimedia. In A. B. Tucker (Ed.), The Computer Science and
Engineering Handbook (pp. 1551-1570). Boca Raton, FL: CRC Press.
Boy, G. A. (2007). Perceived Complexity and Cognitive Stability in Human-Centered
Design. Paper presented at the HCII'07 - HCI International 2007 Conference,
22 – 27 July 2007, Beijing, China.
Daniel, B. K. M. (2007). A Bayesian Belief Network Computational Model of Social
Capital in Virtual Communities. Unpublished Research, PhD Thesis,
University of Saskatchewan, Canada. Retrieved 01/08/2007, from
http://library2.usask.ca/theses/available/etd-07132007-
141903/unrestricted/ben_m.pdf.
Isaacson, C. (2003). The Way of Yoga. London: Element Books.
International Standards Organization. 1994. Ergonomicrequirements for offi ce work
with visual display terminals.Part 11: Guidance on usability (ISO DIS 9241-
11). London: International Standards Organization
Lambropoulos, N. (in press). Preparing the On-line Learners: Information Provision
and Intention Setting by Chatbots. In P. Shank (Ed.), Online Learning Idea
Book. Hershey, PA: Idea Publishing.
Lee, J., McKendree, J., Stenning, K., Cox, R., Dineen, F., & Mayes, T. (1999).
Vicarious Learning from Educational Dialogue. Paper presented at the
CSCL'99 - Computer Supported Co-operative Learning, 11 - 12 December,
1999, Stanford University. Retrieved 12/12/2005, from
http://portal.acm.org/ft_gateway.cfm?id=1150283&type=pdf&coll=portal&dl=A
CM&CFID=15151515&CFTOKEN=6184618.
Preece, J., & Maloney-Krichmar, D. (2003). Online Communities. In J. Jacko & A.
Sears (Eds.), Handbook of Human-Computer Interaction (pp. 596-620).
Mahwah: NJ: Lawrence Erlbaum Associates Inc. Publishers.
Silius, K., Tervakari, A.M., & Pohjolainen, S. (2003). A Multidisciplinary Tool for the
Evaluation of Usability, Pedagogical Usability, Accessibility and Informational
Quality of Web-based Courses. Paper presented at PEG’03, 11th
International PEG Conference: Powerful ICT for Teaching and Learning. 28
June - 1 July 2003 , St. Petersburg, Russia. CD-Rom.
Wegerif, R. (in press). Dialogic, Educational and Technology: Expanding the Space
of Learning. New York: Springer-Verlag.
Wright, S. P. (2004). Exploring Psychological Sense of Community in Living-Learning
Programs. Unpublished research, PhD Thesis, University of Maryland.
Zaharias, P. (2004). A Usability Evaluation Method for E-Learning Courses.
Unpublished Doctoral Dissertation at Athens University of Economics and
Business
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 136
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Main Study:
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 137
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
6.1 INTRODUCTION
The aim of this study is to address the Greek teachers’ passive participation
in the e-learning environment provided by the Greek School Network
(Moodle@GSN). Because the Moodle@GSN technicians were reluctant to
implement the new tools, these tools were incorporated in an experimental Moodle,
version 1.4.5 as with Moodle@GSN. The online course was on the ‘Use of New
Technologies for Educational Project Management’, started on 01/03/2007 and
finished on 31/03/2007. I participated as the 5th e-tutor (Figure 6.1.1-1):
A link in Moodle@GSN in the last week led the participants to the research pool.
Thus comparison was feasible between the two contexts, Moodle@GSN and the
experimental environment. The e-research methodologies used to aid e-learning
engineering were centred on ethnotechnology: fieldwork, logging, thematic and social
network analysis, and questionnaires. More specifically, the next section in this
chapter describes demographics, conditions of working and learning over the
Internet, and examples of Collaborative e-Learning Episodes. Then, it explores the
attributes of the Sense of e-Learning Community Index: community evolution, sense
of belonging, empathy, intensity, trust, e-learning quality; in addition, global and local
social network analysis will aid in investigating group cohesion and the importance of
social networking and collaborative e-learning. The evaluation of the new
collaborative tools is conducted by utility and usability evaluation. Lastly, intervention
analysis examines the overall success and compares the findings with other
contemporary research.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 138
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The initial email call for participation in the online course attracted insufficient
response and another call was issued. To improve communication flexibility, the
participants were given gmail accounts. The course had to be expanded by one more
week than initially planned; this resulted in having the experimental session the week
before Easter 2007. It had 4 sections, project management, blog, wiki, and
videoconference. The participants had to study the educational resources, create
their own context using the tool and evaluate its use. The evaluation of the
videoconference was conducted in the research pool after modifications to the
interface based on previous research carried out before hand into the effectiveness
of the system. Because interface and pedagogical modifications (Delich, 2006) as
well as methods and tasks (Draper, 1993) are interlinked, a decision was made to
change from an instructional (e-tutoring) to a student-centred )collaborative e-
learning) approach.
Two questionnaires were sent: one at the beginning of the course to gather
information about demographics and some background and one at the end with
questions related to the exploratory research questions. One hundred and seventeen
(117) participants returned the first and 59 the third questionnaire. Only 40
questionnaires from the participants who returned the first and third questionnaire
and appeared on both Moodle@GSN and the research pool were accepted for
analysis (Table 6.1.1-1):
Forty questionnaires were selected for analysis using the above criteria and were
given a unique number. Considering the total absence of activity and the ratio
referred to in the literature (20% lurkers’ response), the response ratio compared to
the initial expression of interest (117) was satisfactory (34.1%).
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 139
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Even though 2.03% of the overall responses in the questionnaires items were
missing, this was a low percentage and did not impose any serious problems with the
subsequent analysis (Tabachnick & Fidell, 1996). When participants had failed to
supply a response this was treated as a no response rather than using the mean.
Correlation analysis was used to suggest the highly correlated factors in order to
integrate them in one variable. These results, the small initial sample in the trials and
previous absence of variables justification in the literature suggested that the
questionnaire items could remain the same. Hierarchical Clustering Explorer 3.0 ∗
(HCE) (Seo, 2005) was used for the examination of the data (Figure 6.1.1-2):
Figure 6.1.1-2. Normality overview for tools pedagogical usability and utility in HCE
The responses from the questionnaires were also screened for normality skewness
and kurtosis in HCE (e.g. Hair et al., 1998) to determine whether the variables are
“normal” enough to be analysed; skewness and kurtosis suggested that original
variables can to be used. (Tests of significance for skewness and kurtosis test the
obtained value against a null hypothesis of zero for a normal distribution.)
∗
HCE is a visualization tool for interactive exploration of multidimensional datasets to help users explore
and understand multidimensional datasets by maximizing the human perceptual skills
(http://www.cs.umd.edu/hcil/hce/).
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 140
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
From the initially registered 177 teachers, 49 never fully enrolled. The participants
were asked to create their profiles immediately after their registration. It was hoped
that this would create a co-presence feeling by getting to know each other before
their attempts for active participation. Of the remaining 128 participants, 36 created
illegible profiles due to Greek language encoding, and 95 participants produced
usable profiles. Most of the participants with Moodle profiles were primary school
teachers (n=36, 36%), and IT secondary school teachers (n=22, 28%). Based on the
initial questionnaires (N=40) there were 9 female (n=9, 22.5%) and 31 male (n=31,
77.5%) Greek teachers. The age range in the closed questions was created
anchored in the way teachers are hired and previous studies (e.g. Hlapanis &
Dimitrakopoulou, 2004), and was successfully tested in the preliminary studies.
The analysis showed 1 participant between 20-30 years old (n=1, 3%), 14
between 30-40 years old (n=14, 36%), and 24 more than 40 years old (n=24, 61%) (1
missing). They participated from different parts of Greece, 16 from Athens (Figure
6.2.1-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 141
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
ICT in Education: Most of the participants have worked in the Greek education
system for more than 6 years (N=40). There were 2 new teachers (n=2, 5%), 10
teachers working for 6-10 years (n=10, 25%), 18 working for 11-20 years (n=18,
45%) and 9 more than 20 years (n=9, 22.5%). The participants have used computers
in education: 3 for 1-5 years (N=3, 7.5%), 14 for 6-10 years (n=14, 35%), 15 for 11-
20 years (n=15, 37.5%), and 7 for more than 20 years (n=7, 17.5%). However, their
familiarity with Learning Management Systems (LMS) is relatively low; the majority
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 142
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
have not used LMS at all (N=22, 55%) and 3 for 1 to 6 months (n=3, 7.5%). Only 13
participant have been using LMS more than one year (n=13, 32.5%) whereas one
participants said that she was using LMS for more than 6 years (i.e. less than 6 years
ago). This response may not be reliable since Moodle was only made available to the
public in November 2001 (Moodle e-learning history, retrieved 17/07/2007, from
http://docs.moodle.org/en/Online_Learning_History). The correlations between these
factors are (Chart 6.2.2-1):
100
Number of Participants
22
18 Years in employment
13 14 15
10 10 9 Years using computers
7 Years using LMS
3
2 2
1 1
0 years Months 1-5 years 6-10 11-20 20+ years
years years
Time Scale
The previous graph presents that the use of LMS starts in early employment. This
result may be related to the short time LMS in education has been available; for
example Moodle@GSN was available to the public in August 2002. The use of
computers is in parallel with the years of employment. As for training on Information
and Communication Technologies (ICT) in education, most of the participants did not
attend courses in higher education but outside the formal Greek education system
(n=29, 72.5%). The vast majority did not have any training in LMS (n=26, 65%); 8
learned to use Moodle in an ICT course (n=8, 20%), and 3 within the official Greek
education system in graduate or postgraduate courses (n=3, 7.5%); one of the
respondents had 6-10 years and 2 had 11-20 years in education. As they were not
recent graduates, this means that their course was part of a life-long learning course
whereas LMS were not part of the curriculum. Lack of training and launching online
courses as initiatives on an individual basis has been reported as an e-learning
quality blocking factor (Fernandes & Montalvo, 2006).
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 143
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Familiarisation with the social aspects of the Internet: The majority of the
participants used the Internet on a daily basis: 13 are on-line all day (n=13, 32.5%),
23 once or twice a day (n=23, 57.5%) and 2 every 2 or 3 days (n=2, 5%). Before the
study, the respondents seemed to think that the use and importance of profiles and
discussion forums are significant in e-learning; a small ratio responded on the Likert
scale as low and very low although the ratio of not answering these questions was
relatively high (n=8-13, 22–32.5%). Most of the respondents considered the use and
importance of profiles and forums is of average importance (n=4-9, 10-22.5%),
relatively high or very high: 14 participants have used profiles before (n=14, 35%)
whereas 18 think they are important in e-learning (n=18, 45%); 21 participants have
used forums (n=21, 52.5%) and the vast majority believe in their importance in e-
learning (n=26, 65%). Despite the fact that the participants use the Internet, and
because the Internet is mostly based on textual communication, it appears that they
might not know ways to represent themselves and talk in online discussions.
Overall, the Greek teachers use the Internet regularly and participate in discussion
forums. On the significance of co-presence and e-learning management, they think
that profiles and discussions forums are of average importance. Their opinion was
checked in the final questionnaire and appeared to explicitly correlate communication
and the creation of profiles. Lastly, formal pedagogical training on ICT and LMS is
lacking. These results were similar to my observations and Tsetsilas’ findings (2006)
on Greek teachers’ lack of soft skills, lack of opportunities and professional training
incorporating current pedagogical approaches. In addition, Hartley (2007) suggested
the need for improving British teachers’ social and learning skills, self-instruction, and
their presentation on the web from a life long learning perspective. These results are
also reflected to the next findings.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 144
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Yes No N/A
Freq. % Freq. % Freq. %
Do you know...
1 collaborative ways for e-learning? 11 27.5 20 50.0 9 22.5
Do you think that…
2 your participation is useful? 26 65.0 0 0.0 14 35.0
3 active participation is necessary? 31 77.5 0 0.0 9 22.5
The previous table presents that only 11 participants knew some e-learning
collaborative techniques (n=11, 27.5%); however, most of them either said they did
not know or did not answer (n=29, 72.5%). As for their participation, most of them
said that their participation is useful (n=26, 65%); a significant number (n=14, 35%)
did not answer the question. The majority replied that participation in e-learning is
necessary (n=31, 77.5%), none denied that participation is not necessary, and 9
participants did not answer the question (n=9, 22.5%). The N/A response on
collaboration and participation as well as on the use and importance of profiles and
forums is relatively high compared to the response on questions about personal
details (n=1-3, 2.5-7.5%).
Overall, despite the fact that the Greek teachers did not know any particular
collaborative e-learning techniques, they believed that their own and other e-learners’
active participation is of great importance. This implies a gap between knowing the
importance of participation but not being able to act upon it because of lack of know
how. This result is similar to other results (e.g. Beaudoin, 2002; Gulati, 2006) and this
lack of knowledge of how to work and learn together is inherent in the teaching and
learning modes for teachers’ training found in the literature review. This was found to
be the major difference between the online communities and the e-learning
communities, as the e-learning participants have a specific purpose: to learn. For this
reason, the e-learners were given explicit guidelines on: netiquette, the use of
emoticons, how to write online messages based on results from this project’s
preliminary studies; and the structure of the collaborative e-learning episode (see
Appendix X).
On the question about collaboration with the Greek educational authorities everyone
agreed on the absence of communication channels (Graph 6.2.3-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 145
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
35 34
33
84%
30
82%
25
20
Pedagogical Institute (N=40)
Ministry of Education (N=40)
15
10
5 5
5 13% 13% 2 5%
3%1
0
Some times Very few times Never
The communication channels with the educational authorities are not open as most
participants have never been contacted by the Pedagogical Institute or the Ministry of
Education (average 83%). Some participants (average 16.5%) were approached a
few times by either authorities.
Overall, only two young Greek teachers took part in this study; despite my efforts it
was impossible to find reports on the workforce so no assumptions can be derived
from this result. A reason for this lack of consideration of the age of employment
comes from Katsaros and Karageorgiou on their study on the absence of the Greeks
primary teachers training for the environment in their area between 2001-2006
(2006); the age range had three parameters: up to 40 years old (22 participants), 40-
50 (47 participants) and over 50 years old (21 participants).
The majority use the Internet regularly and have an active online life. However, basic
knowledge of the Internet as a tool for pedagogical activities, how to represent
themselves online as well as techniques to collaborate are all lacking. In addition, the
little communication with the Greek educational authorities implies lack of
participation in any major changes introduced in the Greek education system as
regards the use of ICT in education.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 146
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Views VS Posts: The activity in the two environments was (Table 6.3.1-1.):
Almost double the clicks were related to viewing other e-learner’s profiles and then
discussions and forums (45.1%) whereas ‘forum add forum/post was limited’ (N=175,
0.4%). As stated before, this was deliberately encouraged so that the e-learners
could know more about each other before participation started (Table 6.3.1-2):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 147
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Table 6.3.1-2. Forums and users view log files in the research pool
The logs related to forums (view discussion, view forum, and view forums) in the
research pool were of high priority (1,312 logs, 90.7%) whereas the users view was
limited (69 logs, 4.8%). This also means that the social reasons for passive
participation were almost eliminated at the beginning of the course.
Next, the total number of posts, and posts for analysis appeared as follows (the total
number of posts including my posts is in the parenthesis) (Table 6.3.1-3):
NUMBER OF POSTS
Percentage
Frequency Average per day
(N=714)
Moodle@GSN
1 (31 days) 616(850) 86.3 19.8
Message for Analysis 175 24.5 5.6
2 Research pool
98 13.7 14
(7 days)
Message for Analysis 80 11.2 11.4
Total 714 23.0 (31 days)
Total for Analysis 255 35.7 8.2 (31 days)
The previous table presents that the overall average posting per day was 19.8
messages for Moodle@GSN and 13.6 for the research pool. From the overall 616
posts in Moodle@GSN, 175 (28.4%) were suitable for analysis, whereas from the 95
messages in the research pool, 80 (84.2%) were analysed. A comparison is
presented (Graph 6.3.1-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 148
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
200
175
180
Number of Messages 160
140
120
Moodle@GSN
100
80 Experimental Environment
80
60
40 24.5
20 11.2 11.4
5.6
0
Analysed Messages Percentage on total Average per Day
number of messages (N=31)
(N=714)
Graph 6.3.1-1. Comparison between sent messages and messages for analysis
Temporal View: A temporal overview of all activities can provide an in-depth view
(Table 6.3.1-4):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 149
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The previous table presents that the number of all activities in Moodle@GSN is
decreased as the course progresses. From almost 4,000 activities on the first day of
the course, there were 176 activities in the end of the course. Posted messages
appear as follows (Table 6.3.1-5):
Similar to activities, the posted messages decreased towards the end of the course
with some more messages in the research pool. There were 38 messages on the first
day of the course and 5 messages on the last day of the course in Moodle@GSN. As
the course continued in the research pool, there were 16 messages on the first day
and 29 messages on the last day of the course. A comparison between activity and
posted messages reveals the following (Graph 6.3.1-2):
10,000
3,821
2,088 2,427
1,243 1,553
1,000 866 892
540
Logs
100
38 33 32 34
18 15
10 11 11
Activity
1 Posted messages
01/03/07
03/03/07
05/03/07
07/03/07
09/03/07
11/03/07
13/03/07
15/03/07
17/03/07
19/03/07
21/03/07
23/03/07
25/03/07
27/03/07
29/03/07
31/03/07
Observation dates
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 150
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Activity was significantly decreasing whereas posting was slightly rising in the
research pool. In fact, the posts on the last day (34 messages) were almost as many
as on the first day of the course (38 messages). Because the messages were related
to each topic and were not on social networking (e.g. saying goodbye or staying in
touch after the course), it might indicate that the e-learners had acquired enough
knowledge to finish the course with a good rate of quality posts. Another explanation
may be the Hawthorn effect.
6.3.2 On Participation
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 151
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
the participants in both environments there were about 20% (N=14) more active in
Moodle@GSN. The next sections will try to shed more light in understanding
differences in active and passive participation as mere calculation may not provide a
coherent view.
There were two phases in determining active participation levels as the initial
calculation was found not to be practical on a large scale.
The previous table presents that the proposed model calculated on the total number
of sent messages was not functional on a large scale as all participants appeared to
be on the low activity level.
Active participation Levels II: A second attempt was made to calculate the
messages based on the highest respondent’s posts; as previously, the calculation
was anchored in the bell curve to determine low, medium and high participation
levels (Table 6.3.2.2-2):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 152
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The second attempt provided better practical results, There were 32 low, 1 medium
and 1 active participants in Moodle@GSN, and 16 low, 5 medium and 5 high
participants in the research pool. The highest poster sent 152 messages in
Moodle@GSN (P52), and 12 posts in the research pool (P50). Both medium (P58)
and high (P52) participants were e-tutors in Moodle@GSN. However, P58 posted 1
message and P52 did not post any messages in the research pool. With 12 posts, e-
learner P50 was the highest poster in the research pool, whereas he was a low
activity poster in Moodle@GSN sending 14 messages. P50 posted almost the same
number of messages in 31 days at Moodle@GSN and in 6 days at the research pool.
This means that P50’s posting activity was accelerated from low to high activity in a
week.
Overall, it appears that Moodle@GSN required significantly more e-tutoring than the
research pool and this was reflected in e-tutors and e-learners’ behaviour. Since
there was no e-tutoring in the research pool, active participation was e-learners
directed. Once more it appears that the initial ‘social’ kick on the participants and
instructional modifications to adjust e-learning behaviour to the new collaborative
tools worked. Lastly, the e-learner-generated text provided the context to some
participants to express themselves.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 153
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
participation is when a participant appears to visit the course but never crossed the
threshold of participation. The course lasted 31 days; this means an average of 10
viewing days for each level. Tracking the exact hours and minutes spent on the
system was found to be extremely difficult. (There is a need to note that the graph for
the activity overview in Moodle was misleading, so all participants’ logs needed to be
checked and verified from the individual logs.) (Table 6.3.2.3-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 154
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
There were 2 Moodle@GSN low and high passive participants who posted 1
message each in the research pool. There were 3 low, 2 medium and 3 high passive
participants who posted in Moodle@GSN but not in the research pool. The tendency
of passive and active participation can be depicted in Graph 6.3.2.3-1:
Graph 6.3.2.3-1. Active & Passive Participation locus from the same participants
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 155
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The previous graph presents that low, medium and high level passive participants
remained in the same level of active participation. In other words, a person who
observes a lot of participation is highly likely to be an active participant in a different
forum provided favourable circumstances to the e-learner allow the change of
behaviour. It also depicts the idiosyncratic character of the individual as found in
Nonnecke and Preece (1999), and stresses the importance of investigating the space
between passive and active participation if more participation in the e-learning
communities is desirable. This gap was found as the sleeper effect in the literature
review (Chapter 2.2.1).
Because the result of the increased passive participation in the research pool was
unexpected, I contacted the passive participants and asked them why they did not
participate. It appears that there were different reasons. P1 complained that time was
limited for him (personal communication via email 27/03/2007). P38 said: “I had
extreme difficulty with communication, I did dedicate many hours to build my profile
and in the end I didn’t make it. I couldn’t participate in any of the videoconferences
due to the Internet connection. Once I was asked to prepare a text and I wrote about
the water. I did receive the newsletters from time to time explaining exactly what I
had to do; however, I couldn’t follow either because I couldn’t find the specific link or,
when I could find the link, I couldn’t get into the course and I was wondering in the
classes.” (Personal communication via email, 01/08/2007). Looking into P38’s first
questionnaire, it appears that she was using computers for 5 years; she was not
trained, and never worked with LMS. O1 quit because of GSN technical problems,
and he got scared of the environment and the immediate responses of co-learners;
he said he needed to follow at own pace. Although O2 also quit, he sent a message
addressed to me in Youtube and said the following: “Professional work, kindness,
positive thinking was beyond any expectations. I was proud to be a Greek teacher;
however, I was angry because the Greek educational community does not take
advantage of its talents and knowledge of technology in many fields.”.
P51 visited the research pool for one day and never looked at Moodle@GSN. P1,
P19, and P51 reported that they did not have any spare time. P49, an active
participant in Moodle@GSN, appeared to have serious login problems in the
research pool and the log showed continuous failed logins, I contacted her and tried
to solve the technical problem (28/03/2007). The online course was extended by one
week, the week of participation in the research pool was the week before Easter; this
meant that participants were preparing for Easter or as in the case of P16 were in
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 156
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Europe on Comenius projects. These participants were not in fact passive. When the
participants were asked about the worst thing in the course, most said lack of time
(N=40, 12, 30%), technical problems (10, 25%), and being unable to participate as
much as they wanted (4, 10%). Overall, the main reasons were lack of knowledge
(both personal and technical) as to how to collaborate online (Table 2.3.2.2-1), and
perhaps this prevented the participants from engaging in the way they wanted.
This section discussed active and passive participation; the overall participation was
decreasing and increased slightly at the end of the course. Active and passive levels
of participation provided a technique for coherent and accurate measurement that
can be used by the e-tutors to support the e-learners. Furthermore, the messages for
analysis in the research pool were found to have more replies than the ones in
Moodle@GSN. One way or another, numeric analysis found inadequate for in-depth
messages analysis and as a means to determine their quality. This was however
feasible with the Collaborative e-Learning Episodes analysed in the next section.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 157
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Messages analysis was used to determine the number and quality of the
Collaborative e-Learning Episodes (from now on CeLE) and aimed to:
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 158
In
t ro
Ph.D. Thesis
du
cti
Ti on
m (0
So et 1-
cia ab
le 21
1000
10000
1
10
100
ln (0 /0
et 1 3/
wo – 20
rk 28 07
ing /0 )
M (0 3/
oo 1 20
-2 07
dl
e 4/
03 )
Pr (0/2
Pr ob 1 -0
00
oje lem 8/
ct
sA s (0
03 7)
/2
rc 1- 00
hi 0 8/ 7)
ve 05
( /2
B l 01- 00
og 07 3)
s( /0
01 3 /
To -3 200
Bl o 1 /0 7
og ls 3/ )
& (0 2
al (1 2
Pr 8-
ob 24 0 0 7
lem /03 )
s /2
Page 159
G r (1 00
Pr ou 6-2 7)
oje p 2
Tools and Evaluation Techniques for Collaborative E-Learning Communities
ct s /
id (1 0 3 /
ea 8- 0
s 23 7)
(0 /0
1- 3/
Ot 07
h e 0 6/
0 )
r( 3/
0 2
VC 1- 0
in N e 18/ 07)
0
E- ws 3/
lea
rn ( 2
20
07
8-
in
g 3 0/ )
(2 03
8/
03 /0
7)
-0
1/
Chapter 6: Main Study
04
/0
Correlations between Messages for Analysis, Richness of Text, & Discussion Depth
7)
Richness of Text
Graph 6.4.1-1. Comparison between messages for analysis, richness of text, and discussion depth
Messages for Analysis
Density was calculated with Fahy and colleagues’ density formula, 2a/N(N-1)
(a=interactions, N=number of participants). Messages density was almost double in
Moodle@GSN (0.19) compared with the research pool (0.1). However, Fahy’s
formula does not consider interaction time, and is highly sensitive to the size of the
group. Quantitative variables can give a specific picture of the research context;
however, there is a need to investigate its quality.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 160
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
CeLEs in the research pool were more than twice as rich as in Moodle@GSN, with
3,941 words (30%) in Moodle@GSN and 9,378 words (70%) in the research pool. It
appears that the richest CeLEs last on average 4 days and in 2 occasions the richest
CeLEs were open 10 and 11 days rather than the average of 4 days. They become
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 161
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
richer as the course is unfolding related to the number of days and the number of
words. For example, the CeLE-I lasted 5 days and had 601 words; and more CeLEs
appeared towards the end of the course. It is interesting to note that there is a gap
after the first week; the CeLEs stopped on 06/03/2007 and appeared again on
18/03/2007. The CeLEs’ temporal overview is presented next (Table 6.4.2-3):
18 18 22
16
10 10 11 10
9 10
8 7 8 6 7 8
6 6
5
4 5 4 5
4 4 5
4 5
3 3 3 3 3
1 1 1 1 1
CeLE-I CeLE-II CeLE-III CeLE-IV CeLE-V CeLE-VI CeLE-VII CeLE-VIII CeLE-IX CeLE-X CeLE-XI CeLE-XII CeLE-XIII
Duration 5 1 1 1 11 1 3 10 3 3 4 4 5
Words 593 612 127 183 1,700 517 725 2,204 882 941 3,904 447 135
Threads 10 8 7 5 18 6 3 18 10 8 22 5 6
Posters 4 5 4 4 8 3 3 9 7 6 16 4 3
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 162
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The previous graph presents that there are interrelations between the factors in a
successful CeLE. Since a CeLE is related to co-creativity and new knowledge
construction, idea generation is found to be related to all factors. In other words,
when duration is in days, number of words, threads and individual posters reach a
peak, and idea generation reaches a peak.
However, the aim of collaborative e-learning is e-learners’ generated text; for this
reason the same analysis is presented with and without e-tutors’ contributions (Table
6.4.2-4):
It is evident that the e-learners were more active in the research pool. In
Moodle@GSN there were 2,246 words from the e-learners (53%) and 1,695 (47%)
from the e-tutors. As for the research pool, there were 8,389 words from the e-
learners (89%) and 989 (11%) words from the e-tutors. This difference is significant
as illustrated in the next graph (Graph 6.4.2-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 163
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The previous graph presents that the e-learners’ ratio of words in posting was
increased by 32% in the research pool CeLEs. To conclude the section on the
qualitative variable, 2 CeLEs were selected, from Moodle@GSN and the research
pool. The two environments were viewed as complementary so selection was based
on the most interesting and representative examples, one from each environment,
CeLE-III from Moodle@GSN and CeLE-IX from the research pool. CeLE-II and
CeLE-IX found to be similar as regards the number of words as well as the number of
e-learners and posters. In addition, CeLE-VIII seems to be interesting for
investigation since there were 10 messages sent by 10 different individuals, all being
low active participants. The last CeLE detailed analysis is presented in Appendix X
(A_X_1-4).
In summary, information exchange in CeLE III had a linear structure which unfolded
as a problem solving activity on a technical problem. On the other hand, CeLE IX did
not have a linear structure and referred to the use of specific e-learning tools in
building a project. In CeLE IX there were more agreements, disagreements,
arguments, and exploration of each other’s ideas. These were expressed both as
monologues as well as threaded dialogue. These processes can be seen as
knowledge internalisation and externalisation through monologue and dialogue which
promoted participants’ critical and creative thinking (Appendices A_X_5 and A_X_7).
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 164
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
As there was only one participant in a medium activity level and seven participants
on a low activity level, the appearance of these structures with the aid of previous
knowledge on collaborative e-learning and even the limited use of MessageTag may
be reasons for tackling passive participation (e.g. Klemm, 1998; Khine et al., 2003;
Jeong & Davidson-Shivers, 2006). The use of tools to reveal these structures even
on a limited level, can be indicated by the fact that previous knowledge existed in
Moodle@GSN; however, the CeLEs were not as rich, varied and consistent as in the
research pool. Knowing, viewing, and carefully using the collaborative e-learning
technique can give confidence to and encourage e-learners in their participation. It
appeared that there were transitions between internalisation and externalisation of
learning and this means that design should explicitly support dialogical sequences by
broadening and deepening this space.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 165
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Quantitative, thematic, and social network analysis were employed within the
ethnotechnological framework to evaluate the SeLCI. As before, the questionnaires
from 40 participants (N=40) were analysed. The data for quantitative analysis were
inserted in the Statistical Package for the Social Sciences (SPSS 11.5) and Microsoft
Excel. Thematic analysis was conducted for the open questions and the collaborative
e-learning episodes using Atlas-ti™. Finally, social network analysis was aided by
UCINET (Borgatti et al., 2002).
The next sections will explore the SeLCI starting with the community evolution.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 166
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Evolution: Almost all participants (n=38, 95%) thought that the community was
developing; 2 responses were N/A (not applicable) so there was not an opposing
opinion. As regards the time needed for community evolution, most of them (n=17,
42.5%) said it needs some time to develop: 2 weeks, 13 (32.5%) 1 week, 10 about 4-
5 days (10%) and the rest 6 (15%) 1-3 days. If collaborative e-learning started
developing after the first week, this means 1 to 2 weeks is necessary for developing
a community and collaborative e-learning where the sleeper effect is considered to
occur. In other words, the first week the participants explore and familiarise
themselves with the system and the other learners and make decisions upon passive
or active participation.
11% 7
10% 6 Increasing number of
6 participants
Communication – general
6% 4
4 Collaborative atmosphere
5% 3
E-learners’ participation in
1 2 planning
3% 2 Sense of belonging
2
Personal messages /
2 experiences / first name
1 New colleagues
2% 1
Photos
1
1 Visiting others’ web pages
1
Knew nobody initially
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 167
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The previous graph presents that the most important community evolution elements
were the increasing participation based on mutual help (n=12, 19%) as well as
increasing communication outside and after the course (e.g. phone, SKYPE, blog,
Facebook; n=8, 13%); then shared goals and interests (n=7, 11%) communication
during the course (n=6, 10%); affective elements such as trust and support
(n=6,10%); the quick familiarisation with the e-learning environment (n=4, 6%), the
increasing number of participants (n=4, 6%); the level of communicative activities
(n=3, 5%); and from 2 responses (3%) the collaborative atmosphere, participation in
course planning as adjustment to particular circumstances, the sense of belonging
and expressions of familiarisation e.g. exchange of personal messages, experiences,
and address of e-learners by their first name (n=2, 3%). Other elements (1 response,
2%) were on meeting new colleagues, the number of photos in profiles, visiting each
other’s personal web pages and adding each other’s links, some people did not know
anyone initially, and the quantity and quality of participation.
When they were asked whether they would continue their collaboration outside the
course, 24 said yes (60%), nobody said no; however, there were 16 N/A and missing
responses (40%). Willingness to keep the community going was also evident on their
demand for a blog, a Facebook group be notified if a new course will start. However,
because this is not organised by GSN and the online course was part of this
research, so this request was not feasible.
The increasing interactivity and participation indicated that the community was an
evolving organic entity. This means that community evolution can be moderated by
supporting communication and participation in collaborative activities by helping e-
learners becoming passive and then active participants. For example, the use of
profiles and the initial social interactions to enhance trust and empathy supported e-
learners’ quick familiarisation with each other as well as with the LMS. Having
common targets and discovering similar interests creates a tendency known as self-
disclosure reciprocity as the participants exchange personal information and
experience (Wallace, 1999). The use of other communications means (e.g. phone,
SKYPE, blogs, and Facebook) other than the tools inside the e-learning system has
been found crucial for the community maintenance (Boase et al, 2006), and indicates
a healthy practice in this study. This may also suggest that different people have
different desires when it comes to communication modes.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 168
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Shared Interests & Values: Shared interest was found to be the third community
evolution element. Moreover, most participants replied to the closed question that
they had same interests (n=35, 87.5%), 2 denied (5%) and 2 were N/A (5%). Most
also said they had shared values (n=32, 80%), although 1 denied (2.5%) and 7 were
N/A (17.5%). Following the literature (e.g. Preece, 2000), shared values and interests
is the most important indicator to identify whether a community existed or not. In this
study these results were more than 80% which means that there was a strong
common ground from the participants’ viewpoint, an essential element for
collaborative e-learning.
Knowing the community: The participants said the following about the community
(Table 6.5.1-1):
The respondents liked working together (n=33, 82.5%) and were expressing
themselves freely (n=29, 72.5). The LMS provided the platform for the community to
exist (n=35, 87.5) and the new tools significantly helped in this process (n=33,
82.5%). Most of the participants thought that they knew about the netiquette (n=31,
77.5%) as well as the community (n=25, 62.5%). On freedom of expression the
results were lower (n=11, 27.5%) as well as active participation (n=7, 17.5%). Also,
half of the participants knew nobody before the course and the other half knew a few;
28 (70%) said they developed online relationships, 10 (25%) did not (2 were N/A).
Although the majority of the participants did not have experience in e-learning
communities and online collaboration, they were positive on collaboration and the
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 169
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
only thing that they were not happy about was not to be able to participate as much
as they wanted. Using tools in e-learning was important to them; knowing what the
community was about and developing relationships was a strong determinant in
SeLCI.
New members’ contributions: The results from the closed question on new
members’ contribution were in favour of new members: 34 (85%) supported their
contribution, 6 (15%) were N/A and nobody opposed. Thematic analysis on the e
open question also revealed the following (Graph 6.5.1-2):
The respondents believed that the new members can regenerate the community by
bringing new ideas (n=16, 24%) to share with the older members (n=9, 14%) via
active interaction and participation in the community (n=6, 9%). The most important
element is their enthusiasm (n=5, 8%). The participants believe that heterogeneous
groups function better (n=4, 6%) as they bring up questions (n=4, 6%). Other
responses refer to the need for training (n=3, 5%) and to keep the community going
(n=3, 5%); from two responses each (n=2, 3%) are freedom of expression, bringing
new perspectives and abilities into the community, provide feedback for older
members, whereas willingness to learn and collaborate is essential as well as the
use of new technologies. Shared interest, exploration with propositions and criticism,
quality and life-long learning are the last elements on new members’ contribution.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 170
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Additionally, two issues were revealed with regard to the new members in an e-
learning community. Despite the evidence on the importance of newcomers bringing
new ideas to regenerate the community, there was only one young participant in the
study; however, there were older newcomers. This means that the Greek teachers
even as newcomers have not recently graduated so there is a gap in their
professional training especially when there is a demand for faster training cycles in
recent years. In other words, the lack of training in the use of ICT in Education and
knowledge of collaborative techniques may be due to absence of teachers’ life-long
learning courses in Greece (Tsetsilas 2006). In addition, it is evident that new ideas
are of major importance to keep the community going; this is another collaborative e-
learning beneficial factor to community evolution.
Roles: The participants said that there were roles assigned to the members.
According to the closed question, 23 (57.5%) thought there were roles, 6 (15%) there
were not, and 11 (27.5%) thought there were no specific roles. However, thematic
analysis on the open question revealed the following (6.5.1-3):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 171
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Overall, community management, e-learning and participation were themes for roles
which could be assigned. Equality in participation was also indicated by the fact that
e-tutoring and moderating was evident in Moodle@GSN and did not exist in the
research pool. In other words, the assignment of roles was not found to be a very
strong element in community evolution and collaborative e-learning.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 172
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
(2.5%); and 2 were N/A (5%). In the question on the factors that kept the community
together they said (N=40) (Graph 6.5.2-1):
More than half of the participants (n=22) said that sharing the same goals and
interests was the main reason for holding the e-learning community together (n=22,
30%); then curiosity for new knowledge (n=12, 16%); and equally e-tutors (n=5, 7%)
and collaborative tools (n=5, 7%). More responses were: desire for success and
communication (n=4, 5%), willingness to participation, enthusiasm, participation as
such and effective learning (n=3, 4%), affective elements such as mutual help and
trust, the subject, collaborative learning for projects development and problem
solving (n=2, 3%). Other suggestions (n=1, 1%) were the immediate success and
feedback as well as reliability.
Furthermore, P11 believed that something that shows the sense of community is
active participation despite the technical problems; to P35 there was an ‘incredible
increase of active participation’; to P22, there was ‘willingness to collaborate beyond
and after the online course’, to P39 there was a ‘desire to keep the community going’,
and to P38 and P47 there was mutual help, understanding, and trust. P14 said that
concerning the use of profiles that the ‘photos created a climate of familiarisation with
each other and helped in developing a sense of belonging’.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 173
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Overall, it seems that the same factors that defined the sense of belonging are the
factors that contribute to the e-learning community evolution: shared interests and
values. Also, new knowledge is the collaborative e-learning outcome. The mediators
in this process are the e-tutors and the tools.
100
48
21
Codes
10 10
Responses
6
5
1 1
Community Management e-Learning (35%) Technology (6%)
(59%)
(Percentage on codes)
Community management is considered the most important factor in this study (59%),
then e-learning (35%), and third the tools (6%). In other words, community
management facilitates community evolution and thus allows e-learning to take
place. Learning is an important element of community evolution and vice versa,
meditated by artefacts; in this case e-learning tools (Lave & Wenger 1991).
6.5.3 Empathy
EMPATHY FACTORS
Know what Other Feel what Other was
was feeling when feeling when reading Action
EXTEND reading a message a message
Number Percent Number Percent Number Percent
Very Small 1 2.5 2 5 11 27.5
Small 8 20 6 15 7 17.5
Neither 17 42.5 15 37.5 9 22.5
Great 13 32.5 15 37.5 11 27.5
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 174
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Very Great 0 0 2 5 2 5
N/A 1 2.5 0 0 0 0
SD 0.9 0.9 1.3
Total 40 100 40 100 40 100
On the question of knowing what someone else was feeling (42.5%) replied they
more or less knew what the other person was feeling when reading a message and
13 (32.5%) they knew to a great extent; 8 (20%) said they knew to a small extent, 1
very small (2.5%) and one response (2.5%) was N/A. On “feeling what the Other was
feeling”, the responses were equally distributed between neither great or small and
great extent (n=15, 37.5%); 6 (15%) said they could feel other’s feeling to a small
extent, and equally 2 (5%) were on the very small and very great scale. As for
whether they took any action, 11 respondents (27.5) equally said to a very small and
great extent; 7 (17.5) to a small extent and 2 (5%) took action to a very great extent.
It appears that the results reach a peak in “knowing what the other person was
feeling”, they are more distributed on “feeling what the other person was feeling”, and
they reach a down peak in “action taking”. A scatter plot provided a more detailed
view (Graph 6.5.3-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 175
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The strength of association between the three variables is positive but it does not
suggest a strong linear relationship. A moderate association (r=0.527) is between
“Feel what Other was feeling when reading a message” and “Action” with correlation
coefficient p≤0.01; “Feel Other” and “Action” association was the weakest (r=0.347)
with correlation coefficient p<0.05. Cronbach's alpha, as a coefficient of reliability and
consistency between the variables found α=0.698, which is on the limit of acceptance
in social research (0.70). Although empathy has been related to gaze and body
language (e.g. Lanzetta and Englis, 1989) it seems that it can occur online and it is
influenced by the properties of different communication media (Preece, 2004). It was
not possible to extract explicit relationships between the results of this study and
others as the frameworks for investigating online empathy were different. In this
study, the respondents appeared to know what the other person was feeling when
posting more than actually feel the poster. Nonetheless, in regard to action taking,
people may need time to actively participate in activities (the sleeper effect) before
consciously decide to work with others (the least collaborative effort).
There is a need to note that empathy has been related to the mirror neurons; this
means that the e-learners in their profiles provided information to the mirror neurons
to build representations of the other learners and their actions (Goleman, 2007).
However, investigation of such correlations was not part of this study.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 176
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
6.5.4 Trust
TRUST
Statements Yes Percent No Percent N/A Percent
I can trust most of the
1 37 92.5 1 2.5 2 5
participants.
I have to be very careful as some
2 37 92.5 3 7.5
take advantage of others.
3 Most are trying to help. 38 95 2 5
4 Nobody can trust anyone else. 38 95 2 5
It appears that most of the participants could trust other e-learners with a similar
specialisation to a great (n=16, 40%) and very great extent (n=12, 30%) whereas 11
(27.5%) could trust them more or less; as for individuals who had similar writing
skills, the results indicate trust to a great (n=16, 40%) and very great extent (n=12,
30%) whereas 12 (30%) could trust them more or less. The results on trusting the e-
tutors were the most positive with 39 (97.5%) to trust them on a great or very great
extend and 1 neither a small or a great extent (2.5%). As for trusting the experienced
individuals the results were: 16 (40%) to a great extend, 15 to a very great extent
(37.5%) and 9 (22.5) neither small or great extent. The question to ensure that the
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 177
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
participants did not respond randomly revealed that the majority (n=32, 80%) could
trust other people.
Lastly, all but 1 participants reported that the level of trust had risen (n=39, 97.5%).
Almost all participants felt that they could trust the e-tutors and their co-learners.
Trust has been reported to be related to different forms of awareness, such as
personal information, presence on the community, demographic backgrounds,
capabilities and skills in performing specific tasks (Daniel, 2007:124). Trust is also
related to the mental models people develop when they first meet as well as the
content of their conversation and tend to develop very quickly (Norman, 1988).
However, these suggestions were not explicitly investigated in this study; here, trust
was linked to reciprocity and levels of participation to allow participants to work freely
together, evident in the freedom of expression to a great extent (n=29, 72.5%).
6.5.5 Intensity
Intensity refers to the participation levels and persistence; the latter is the
level to which participants pursue topics.
Participation Levels: Active and passive participation levels were found to vary in
the two environments. Seeing these levels as a process from low passive to high
active provides a different overview for intensity (Graph 6.5.5-1):
35
32
30
Number of participants
25
20
A: Moodle@GSN
16 B: Research Pool
15
10
7
5 5 5 5
4
1 1 1 1
0
Zero Low Passive Medium High Passive Low Active Medium High Active
Participation Passive Participation Participation Active Participation
Participation Participation
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 178
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
PERSISTENCE IN MOODLE@GSN
Depth of persistence
Section Forums
Forums Messages
Introduction (01- 21/03/2007) 43/49 66/231
Timetable (01 – 28/03/2007) 7/7 127/127
A Introduction Social networking (01-24/03/2007) 1/1 30/30
Moodle (01-08/03/2007) 2/2 7/7
Problems (01-08/05/2003) 6/22 117/127
Total 5 59/81 347/522
Project
B Projects Archive (01-07/03/2007) 1/1 6/6
Management
Total 1 1/1 6/6
Blogs (01-31/03/2007) 42/43 187/198
C Blogs Tools (03-12/03/2007) 3/3 9/10
Blog & HTML (05-12/03/2007) 1/1 3/3
Total 3 46/47 199/211
Problems (12-23/03/2007) 9/9 37/37
Design (17-18/03/2007) 0/2 0/6
D Wikis
Practicality (13-18/03/2007) 2/2 10/18
Groups (18-24/03/2007) 6/7 31/31
Total 4 17/20 78/92
Technical Problems (16-22/03/07) 1 12/12
E Videoconferencing
Groups (18-23/03/07) 1 16/16
Total 2 2/2 28/28
F Internet Cafe Project ideas (01-06/03/2007) 1/1 3/3
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 179
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The previous table presents that 59 out of 81 forums and 347 out of 522 messages
were following the subject of discussion in the introductory section; in the project all
messages were on topic these are 1 out of 1 forums and 6 out of six messages; in
blogs, persistence was strong as 46 out of 47 forums and 199 out of 211 messages
were on topic; in wikis the persistence depth was 17 out of 20 in forums and 78 out of
92 for messages; as in videoconferencing persistence was 100%, 2 in 2 forums and
28 out of 28 messages; the same rate appeared in the internet café with 3 in 3
forums and 9 out of 9 messages. The forums overall percentage was 83.1% with 128
out of 154 forums persistence depth which meant that the forums were relevant to
the forum topics. Slightly lower was the messages overall percentage (76.8%) with
667 out of 868 messages to follow the discussion topic. The overall persistence
depth in Moodle@GSN was 80%.
More persistence was observed in messages with 2-3 replies. As for the rest of the
messages, there were a small number of them irrelevant to the topic, sometimes
coming from the same e-learner; for example, 5 out of 6 irrelevant messages in the
‘Problems’ forum came from P48; they were events announcements. (Note that there
were no members’ announcements area in the online course.) Both forums on ‘wiki
design’ were irrelevant; this perhaps was because of the title forum: ‘What are the
main characteristics in wiki design? the goal to match, to serve the audience, both or
something else?’ As for levels of persistence in replies, the persistence depth was
found to be relevant to the thread depth; in other words, the more the replies the
more the probability of lack of persistence. Some times there was a reason for
shifting the discussion focus; for example, in Wiki topic, the focus was shifted in the
practicality discussion when one of the members found to have posted in another
person’s wiki without realising it (Reply 5): “Hi O3, I am P50; without doing it
intentionally, you created a webpage post in my wiki…”. Two similar changes of
focus occurred in the blog/blogs discussion when a problem on the suggested url
appeared and 5 messages were on finding the correct url. Lastly, a large number of
forums indicated a small number of replies so the level of persistence was maximum
for these forums. Overall, shift of focus was justified.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 180
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
PERSISTENCE IN MOODLE@GSN
Depth of persistence
Section Forums
Forums Replies
News (28-30/03/07) 2/2 5/5
A Introduction
VC in E-learning (28/03-01/04/07) 8/8 63/70
Total 2 10/10 68/75
Percentage 100 90.6
Average 95.3
The forums had 100% persistence; with 10 out of 10 messages following the topic.
As for the messages, there were 68 out of 75 messages exhibiting persistence, with
90.6% percentage. The overall persistence in the research pool was 95.3%. In depth
analysis revealed that the discussion on re-using videos in a project in the
videoconferencing forum produced 21 replies; however, after the 13th reply the focus
changed to how the Ministry of Education supports the teachers on the projects and
what happens when the project finishes. Then the focus shifted to the Greek
education system and teachers’ training, and the last two messages partly brought
the focus back to the use of tools in projects.
More intensity found in the research pool: the process of participation was more
stable and coherent, and there was an overall persistence of 95.3% comparing to
80% in Moodle@GSN even though the number of messages was low.
Intensity appeared to be higher in the research pool: the participation levels were
more coherent and stable in the research pool having all levels from low passive to
high active participation; and persistence was found 90.6%. It appears that initial
online dialogical argumentation lacked depth and was redundant as participants
failed to sustain interaction, also found to be evident in the research by Khine and
colleagues (2003); as the participants gained knowledge and experience of
collaborative techniques based on information provision and observation of the e-
tutors, their behaviour changed in the research pool. It is interesting to note that for 3
years there were 19 messages with 1 one reply, whereas 1 message produces 27
replies (Chapter 4.3.1.2). Intensity also provides evidence for establishing common
ground. This is done by checking whether a conversation partner has heard and
correctly understood what is being said (Preece & Maloney-Krichmar, 2003). Lastly, it
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 181
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
is evident that intensity can depict any imbalance or equilibrium between passive and
active participation and a level of persistence as a SeLCI determinant.
The respondents said that they asked help from the e-tutors (n=37, 92.5%) and less
from their co-learners (n=3, 7.5%). The majority replied that they actually learned
ways for collaborative learning (n=37, 92.5%), 1 did not (2.5%), and 2 N/A (5%).
More specifically, they suggested ways to achieve collaborative e-learning (N=41;
one participant provided more than one suggestion) (Graph 6.5.6-1):
Collaborative e-Learning
Number of responses (N=41)
0 2 4 6 8 10 12
37% 11
Collaborative activities
27% 8
7% 2 Mobility of ideas
1
1
Learn to communicate
3% 1
1 Vicarious learning
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 182
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Information and knowledge exchange were the most important factors for
collaborative e-learning (37%); then collaborative activities (27%), dialogue
development (17%), and mobility of ideas (7%). One suggestion each (3%) referred
to learning to communicate, vicarious learning, new skills acquisition, and the number
of created projects. Only 5 responses were on tools, 4 on the new collaborative tools
(80%) and 1 on profiles (20%). It appears that mere information was essential to
trigger collaboration but it was the collaborative techniques and the use of tools that
transformed information to collaborative e-learning by supporting different learning
styles. This result indicates that progressive discourse can be the outcome of
increasing participation in collaborative e-learning communities.
The next graph shows more specific opinions with regard to whom they learned from
(Graph 6.5.6-3):
3, 6%
3, 6%
E-tutors
Otherlearners
More experienced learners
17, 32% 30, 56% Own work
The previous graph presents that the participants learnt from the e-tutors (n=30,
56%), other learners (n=17, 32%) and equally more experienced learners (n=3, 6%)
and on their own (n=3, 6%). It appears that passive participation and vicarious
learning was one of the learning styles; P21 also said that she was ‘watching how
other people were working’. This result is in agreement with the centrality scores and
the responses on passive participants.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 183
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Collaborative e-Learning
100
62
41
17
Codes
10
8 Responses
1
Community Management e-Learning (30%) Technology (7%)
(63%) (Percentage on codes)
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 184
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Two adjacency matrices were produced one 64*64 for GSN (Figure 6.5.5-1) that
included the main participants and others who had been interacting but not on a
regular basis and did not fit the criteria of participants’ selection. Also another 41*41
matrix was produced for the research pool that included my interactions (Valued links
and passive participants zeros are depicted; all other zeros were eliminated to make
the matrix more legible) (Figure 6.5.7-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 185
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Several problems occurred in producing the matrices: one was the discrepancy
between the system reply and the actual address of reply and the other was the
participation of two e-tutors and myself. First, the function ‘Reply’ in Moodle was
referred to the previous message whereas the actual reply could be addressed to
one or more e-learners sometimes higher up the forum. So in this study, where
apparent, the input in SNA was the actual addressed reply as conceived from the
message rather than as depicted on the system; this indicated a limitation in the
forum application in Moodle. For example, someone may send a post later in the
week by replying to the last message; however, the message may have been
addressed to somebody else. Messages that were replied to everyone as when the
participants were introducing themselves were linked to themselves on the adjacency
matrix. Then, two new sets of adjacency matrices were analysed, one with the
researcher and the 2 e-tutors as the most active participants in GSN, and one without
the 3 actors, these were two 38*38 matrices. This decision was made because this
study focuses more on collaborative e-learning for e-learners-generated text rather
than the e-tutors-generated-text.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 186
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
density. The participants actually recognised their limited participation; they said they
were not as active as they wanted to be. In addition, the 2 highest posters in
Moodle@GSN influenced the groups’ density level; this means that the actual
increase in participation was almost doubled (0.0418 - 0.0256= 0.0162). This was
also evident in the collaborative e-learning episodes text richness, as it was doubled
in the research pool (see Table 6.4.2-2).
Reciprocity in SNA is the number of ties that are involved in reciprocal relations
relative to the total number of actual ties (Hanneman & Riddle, 2005). Reciprocity
appears higher within the e-learners (Graph 6.5.7.1-1):
(a) (b)
Graph 6.5.7.1-1. Reciprocal ties in GSN (a) & the research pool (b)
There were 4 reciprocal ties in GSN (28.6%) and 10 (71.4%) in the research pool.
However, due to the e-tutors role in GSN, it appears more as an evolutionary
process.
Reciprocity is also related to the social exchange theory (Blau, 1964); it posits that
individuals engage in social interaction based on expectations or the benefits active
participants can get from active participation, for example some sort of personal gain
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 187
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
A clique is a subgroup, a set of actors with each being connected to each other as a
maximal complete subgraph of three or more nodes (members) adjacent to each
other and there are no other nodes in the network that are also adjacent to all of the
members of the clique (Laghos, 2007). Cliques may overlap, that is a forum member
(node) can be a member of more than one clique (Bock & Husain, 1950). The results
presented in the following table are cumulative and refer to cliques created by 3, 4, 5
and 6 participants (Table 6.5.7.1-2):
CLIQUES
Moodle@GSN Research Pool
Minimum set size
of participants All E-learners All E-learners
6 2 0 0 0
5 33 0 0 0
4 58 0 2 1
3 68 4 15 12
As most the participants did not know each other before the study and were more
skilled in the research environment, the cliques were the glue for forums. However,
what fostered the cliques was not investigated in this study.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 188
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Structural equivalence describes the actors who have similar patterns of relations
to others in the network and exhibit similar communication behaviour. It presents a
different clustering view within a human network. Equivalence is important for
generalizations about social behavior and social structure; actors must not be
thought about as unique persons, but as examples of categories (sets of actors) who
are in some way, "equivalent” (Hanneman, 2001).
Two actors (nodes) are said to be structurally equivalent if they have identical ties
with themselves, each other and all other vertices (de Nooy et al., 2005). It is
computed by the Euclidean distance of tie-value from and to all other nodes (Lorrain
& White, 1971). The CONCOR technique (CONvergence of iterated CORrelations;
White et al., 1976) uses dendrogrammes (tree-diagrammes) for hierarchical
clustering whereas other techniques use algorithms to calculate network members’
individual behaviour (e.g. Everett & Borgatti, 1993). The CONCOR technique
calculates Pearson’s correlation coefficient between columns and depicts whether
two nodes are structurally equivalent if the corresponding rows and columns of the
adjacency matrix are identical. So the degree to which two nodes are structurally
equivalent can be evaluated by measuring the degree to which their columns are
identical. CONCOR is a divisive top-down clustering technique; it begins with one
group and then divides it up so the dendrogramme looks like an inverted tree. This
structure is calculated and thus artificial, resulting in failing to identify observed
clusters.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 189
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
In the research pool 3 participants were active in 2 second level groups and 5 first
level groups. The next CONCOR dendrogramme reveals 7 first and 4 second level
participants (Graph 6.5.7.1-4):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 190
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Here, the ties are less than the one with the researcher and the 2 high participation
e-tutors, however, the overall structure of the groups remain the same. Lastly, the
next dendrogramme refers to the e-learners in the research pool (Graph 6.5.7.1-5):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 191
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
This graph reveals 5 first and 2 second level multi-actor positions with one solo-actor
position.
In conclusion, if grouping actors with equivalent behaviour, the results from the
previous dendrogrammes appear as follows (Table 6.5.7.1-3):
EQUIVALENT E-LEARNERS
Moodle@GSN Research Pool
1st level 7 7 5 5
2nd level 3 4 3 2
Overall, the social network analysis attributes of the Sense of e-Learning Community
Index (SeLCI) indicated that the interactions’ weight (density) was doubled in the
research pool; participants’ preferences (reciprocity) were also significantly
increased; more similar behaviours (structural equivalence) were observed in
Moodle@GSN rather than the research pool; and there were some small groups
(cliques) that remained almost the same throughout the study.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 192
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Centrality is measured by the portion of nodes that are adjacent to each node, the
sum of each row in the adjacency matrix representing the network. The centrality
scores were (Table 6.5.7.2-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 193
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
In this study, the e-tutors’ role was apparent in Moodle@GSN. For the e-tutors the
scores were: out-degree=13.852% (SD=20.321), in-degree=73.97% (SD=13.99). For
the e-learners the scores were very limited: out-degree=10.826% (SD=2.154), in-
degree=5.425 (SD=1.459). The e-learners had more stable scores in the research
pool; with the e-tutors the scores were: out-degree=8.606% (SD=2.316), in-
degree=49.49% (SD=2.089); without the e-tutors the scores were: out-
degree=9.372% (SD=2250), in-degree=6.671% (SD=1.983).
The out- and in- degree centrality appear to have great differences depicted mostly in
the standard deviation rather than centrality itself. Standard deviation dropped in the
e-learners centrality scores in the research pool which means that the differences
between the e-learners were diminishing. There was an increase in in-degree
centrality and a decrease in out-degree centrality. This means that the e-tutors had
the power in the information flow in GSN and the e-learners in the research pool in a
more distributed manner. In a more in depth analysis, the top 10 centrality scorers in
out- and in- degree centrality (Table 6.5.7.2-2) were:
If the 3 e-tutors (P52, R, and P58) are omitted, the next 3 active participants appear
first in out-degree centrality (sent messages) in GSN. P18 and P37 from GSN appear
to be among the top scorers in the research pool. The top scorers for in-degree
centrality were (Table 6.5.7.2-3):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 194
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
As for the received messages (in-degree centrality), the same participants appear to
have incoming messages posted specifically for them; this is more apparent in the
research pool where exactly the same e-learners were in the top 10 as there were
central students controlling in-coming and out-coming connections. In addition, the
increase of in-degree centrality indicates that they were more communicative and
received more messages; the simultaneous slight decrease in the out-degree
messages indicates that they lost some of their power and this power was distributed
to the other e-learners. In other words, the responses were originated from a group of
members that was larger than the group that received the messages. In a way, it
indicated a movement from a powerful group of e-tutors to a group that was working
more and more collaboratively to increase their learning; the active participants
became more democratic in their communication instead of maintaining their status.
Closeness: Nodes with low closeness scores have short distances from the others.
In other words, a node has high closeness centrality if it has very short
communication paths to the others. ‘In-closeness centrality’ is measured as a
function of the minimum geodesic distance from all other nodes to the selected node;
while ‘out-closeness centrality’ is measured as a function of the minimum geodesic
distance linking that node to the other nodes. While degree centrality measures use
only direct and local connectivity information, closeness centrality measures also use
indirect connectivity information (Braha & Bar-Yam, 2004:25). An example for its use
is the following (Borgatti, 2005:59): organizations with low closeness in an R&D
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 195
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
technology-sharing network can develop products sooner than others; a virus can
sooner infect the members with low closeness.
As with degree centrality, with regard to in- and out- closeness, there were major
differences between the e-tutors and e-learners in the two environments. This was
indicated by the closeness score, the means and standard deviation. For example,
including the e-tutors in Moodle@GSN the in-closeness score was 73.97%
(SD=10.762) and the out-closeness score 78.30% (SD=11.387); without the e-tutors
in Moodle@GSN, the in-closeness score was 32.68% (SD=1.859) and the out-
closeness score 55.04% (SD=2.532). In the research pool the results are more
equally distributed and distance of communication was reduced: with the e-tutors the
in-closeness score was 49.49% (SD=5.147) and the out-closeness score 42.74%
(SD=5.088); without the e-tutors the in-closeness score was 49.83% (SD=4.919) and
the out-closeness score 42.53% (SD=4.626).
Overall, ‘in-closeness centrality’ represents the speed of interaction from all other
nodes to the selected node; so in Moodle@GSN this speed was high when the e-
tutors were included; the standard deviation was high as well. ‘Out-closeness
centrality’ represents the speed of interaction from one node to the other nodes; this
score and standard deviation was also high in Moodle@GSN. These scores
indicated that the e-tutors were controlling the speed of information flow. However,
the scores without the e-tutors and in the research pool were more equally distributed
and the standard deviation was very low, indicating a more stable interaction speed
between the e-learners. A reason may be that e-tutors were connected more times
than the e-learners and this affected their interaction speed.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 196
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Overall, the centralisation indexes were: for interaction direction (in- and out-degree
centrality) the in-degree centrality increase and the out-degree centrality decrease
indicated that the e-tutors had the power in the information flow in Moodle@GSN and
the e-learners in the research pool in a more distributed manner. This was also
evident in the interaction speed (closeness) and control (betweenness) that showed
that the e-tutors hold the network power in Moodle@GSN whereas this power was
quite evenly distributed in the research pool and the discussions have not been
monopolised. Moreover, there was no single participant who ranked high in all the
centrality measures in both environments including the e-tutors and myself.
Other than global Social Network Analysis (SNA), local SNA describes the human
and information network in a particular situation. The Visualisation Interaction Tools
Nodes and Centrality (VIT Nodes and VIT Centrality) supported local SNA nodes and
centrality in real time.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 197
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
participants were not given specific information on the exact use of these tools as the
tools should indicate their own use (usability).)
In VIT Nodes the individuals are represented as circles (nodes), the direction of the
messages is indicated by an arrow and the number represents the number of
messages (Graph 6.5.7.3-1):
P37 was the information broker in this CeLE. The reciprocal tie with O2 was an
argument. She also responded to her own message a couple of hours later after the
argument with O2. Most participants were replying to P37 and two of them talked to
each other. It is interesting that this CeLE was developed by different individuals with
only two interlocutors exchanging 2 messages. In other words, the discussion was a
collaborative activity between 7 individuals. VIT Centrality provided a different
viewpoint (Graph 6.5.7.3-2):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 198
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
In VIT centrality P37 is clearly located in the middle of the e-learning social network.
VIT centrality also indicates the response time space related to geodesic distances
between the participants. As a central connector and information broker she moved
the knowledge around leading to a new proposition by taking into account her co-
learners responses even though they appeared as low activity e-learners (i.e. only
O2 was an e-tutor).
To sum up, it appears that after the initial knowledge acquisition and information
exchange, the collaborative techniques and tools helped participants to learn from
the e-tutors, their co-learners and on their own. As they did not have any previous
knowledge of collaborative learning and techniques they acquired this knowledge for
community knowledge building. This means that e-tutors have a complex job that
incorporates moderating as well as e-tutoring (Salmon, 2000). It was also suggested
that knowledge awareness plays a major part in the creation of opportunities for
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 199
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
efficient and effective collaborative opportunities (Ogata & Yano, 2000) which leads
to the fact that different learning styles are supplementary to each other in e-learning
environments.
Lastly, with regard to collaborative e-learning quality, the participants learned from
the e-tutors (instructional learning), the other e-learners (collaborative learning) as
well as on their own how to work together and how to use the new tools and new
technologies (vicarious and self-organised learning). Almost the same number of
collaborative e-learning episodes, 6 in Moodle@GSN and 7 in the research pool, add
to the evidence for the e-learning quality. CeLE was found to build on progressive
discourse and fill the middle space between internalisation and externalisation in the
form of monological to dialogical sequences. This was evident not only within one
CeLE but in the participation process as monological postings stand alone without
open clues for dialogue as in information provision. In a way, e-learners were
progressively adopting a two-way communication. This means that there were more
clues as opportunities for critical engagement in dialogue in the research pool caused
by two events, the initial need for familiarisation with collaborative e-learning in the
early stages of the online course and the sleeper effect, and the use of MessageTag
that revealed the collaborative e-learning structure. Moreover, correcting
communication gaps between the collaborative learning discussion stages is feasible
for the e-learning participants.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 200
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The participants’ responses in the closed question revealed that the new tools
supported the e-learning community development (n=33, 82.5%). The tools were: (a)
Participation graphs and avatars; (b) MessageTag; (c) Visualisation Interaction Tool for
Nodes (VIT Nodes), and (d) Visualisation Interaction Tool for Centrality (VIT Centrality).
The overall results are presented in Appendix XII.
The overall score for pedagogical usability was found to be just higher than the
average of 1 to 5 Likert scale (3.2). Most of the pedagogical usability scores for the
new collaborative tools were satisfactory. However, originality (2.4) and learnability
were low (2.7). The law learnability rating may explain why the tools were
infrequently used (2.4). This result was similar to the preliminary studies although
there the tools were being used by the developers who perceived the tools
differently. Nonetheless, according to the participants, the use of the new tools was
the third best thing in the project after collaboration and feeling a sense of
contributing to something great (Appendix A_XII_4).
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 201
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The tool was not used in the introductory news section and the participants started
using it in the main discussion. The results were very similar to the e-mmersion
block; 20 out of 79 messages were tagged with an average 27.3% use. Comparing
the results with the developers’ results (55.3%) a great difference appears to be
between the way average users’ utilisation of the new tools and the more advanced
users. Involving users with different levels of experience or filtering of users based on
their profile for basic and advanced user testing can provide different results;
advanced users might know uncommon ways to overcome problems on the
interface, get familiarised quickly or be keen to experiment with new tools.
Interestingly, P6 sent a message to the forum saying that the new tools are invisible
and lost in the interface. He said that most users will not notice them and he gave
examples on MessageTag and Visualization Interactions Tools nodes and centrality:
“it is not easy to actually see the links that lead to them because the MessageTag is
at the end of the message and the images for the VIT are invisible”. In addition, his
message indicated frustration concerning their use which is in accordance to the low
learnability score: “At last! I managed to find a way with the VIT nodes and
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 202
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
centrality…” (“Επιτέλους! Έβγαλα μια άκρη με τα VIT nodes & centrality.”). Lastly, he
expressed his enthusiasm about the tools and suggested that their use will be
determined by time.
P6’s messages raises questions on the decisions made about the images selected
after the developers’ recommendations. A developer needs to listen to users’
opinions; however, her decision making is based on her own expertise and usability
tests in usability laboratories, especially when the developers’ sample is relatively
small. (There were 3 Greek teachers who were Moodle developers participating in
tools’ testing.) It also appears that new tools need more than one week of use and
evaluation. However, the use of visualisation interactions tools was not apparent
without the aid of the literature; this is in contradiction to usability rules as the tools
should be easy to use. As tools to represent social networks have now reached the
average user (e.g. different social network applications on Facebook,
http://www.facebook.com), their utilisation will be more obvious and the users will be
more familiar with their practicality.
This section discussed results from the questionnaires, the tools, and the
participants’ comments. The next section refers to the most noticeable correlations
and crosstabulations between the pedagogical usability and utility factors as
appeared in the questionnaires.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 203
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
In more detail, the matrix with the data was exported from SPSS to Excel and then
inserted in the HCE to distil them. In HCE, N/A were viewed as missing data, and all
data were initially filtered with SD≤1. Twenty out of 40 rows remained and then, in
order to increase similarities and reduce differences between the data, the data were
normalized row by row in a scale 0.0 to 1 using the equation . Then Pearson’s r
correlation coefficient was used to identify 1,770 correlations variables with lower
score -0.747 and highest score 0.981 (scores<1). The information visualisation
provided a simultaneous correlations overview, scatter plotting, and in depth analysis
(Figure 6.6.2-1):
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 204
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
HCE provided a correlations overview from the questionnaires (bottom left), scatter
plotting (bottom right), and in-depth analysis of the results. The latter means that
clicking on the participants (top left) located them on the scatter plot; additionally
there was a visual summary (top right).
Not all 1,770 variables are presented here; the variables with r>0.7 (0.977<r>0.703)
and r>-0.600 (-0.600>r<-0.747). As in the previous findings, it appeared that
correlations exist between the two groups of tools the collaborative e-learning and
participation tools (Participation graphs and avatars, and MessageTag) as well as the
Visualisation Interactions Tools Nodes and Centrality (VIT Nodes, VIT Centrality)
used for social network analysis. This can be explained under the rubric of
functionality; the two groups of tools supported different practices in discussions, the
first to structure and analyse collaborative e-learning, and the latter to support
visibility in social networks. In other words, the practicality of the first was more
explicit than the latter.
The positive correlations between participation graphs and avatars, and MessageTag
were: failure (r=0.944); accessibility (r=0.948); educational goals (r=0.883); fast
response (r=0.751); motivation to participate (r=0.71); educational goals and
functionality in graphs and avatars alone (r=0.782); and satisfaction and educational
goals in graphs and avatars alone (r=0.712). The results show that there were
problems concerning tools failure and accessibility; however, there were in
accordance to educational goals, motivated participation, and the e-learners
expressed their satisfaction. This preference was also evident in the average use
(Likert scale 1-5): for graphs and avatars 2.7 (SD.=1.3); and for MessageTag 2.7
(SD=1.2).
The positive correlations between the VIT were: where Nodes failed, Centrality failed
(r=0.977); where Nodes functioned, Centrality functioned (r=0.961); satisfaction
(r=0.955) and motivation to participate was related (r=0.930). Originality was found to
be an issue only for VIT Nodes, related to: educational goals (r=0.81); functionality
(r=0.771); graphics (r=0.752); satisfaction (r=0.721); and support Collaborative e-
Learning (CeL) (r=0.712). In Nodes alone functionality was also found relevant to
satisfaction (r=0.792); motivation to participate and support CeL (r=0.788);
satisfaction and support CeL (r=0.748); and satisfaction and graphics in visual design
(r=0.710). In Centrality alone correlations were found between: satisfaction and
functionality (r=0.801); satisfaction and support CeL (r=0.784); satisfaction and
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 205
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Less strong negative correlations appear only for the VIT. For VIT Nodes the results
were: satisfaction was negatively related to failure (r=-0.0609); motivation to
information overload (r=-0.632); information overload and attractiveness (r=-0.6776);
learnability and attractiveness (r=-0.712); and accessibility and learnability (r=-0.729).
For VIT Centrality the results were: information overload and motivation for Centrality
(-0.628); satisfaction and information overload (r=-0.629); information overload and
attractiveness (r=-0.747). It appears that information overload was the main and
learnability the second pedagogical usability problem that affected attractiveness,
motivation to participate and the overall satisfaction. VIT were not used as much as
graphs, avatars and MessageTag: VIT (Likert scale 1-5): VIT Nodes 2.1 (SD=1) and
VIT centrality 2 (SD=1).
The difference in the standard deviation on the use of tools showed that some e-
learners used the tools more frequently than others. After crosstabulating the first
and final questionnaires, it seemed that the e-learners who did not have any
experience of LMS used the new tools more than the experienced e-learners (Table
6.6.2-1):
Tools
Time Using LMS VIT VIT
Graphs/avatars MessageTag
Nodes Centrality Mean
# # # #
1 1-3 2 2 2 2 2
2 Months 4-6 1 1 1 1 1
Missing 3 3 3 3 3
3
Total 40 40 40 40 40
4 0 22 22 22 22 22
5 1-5 13 13 13 13 13
6 Years 6-10 1 1 1 1 1
Missing 3 3 3 3 3
7
N/A 1 1 1 1 1
Total 40 40 40 40 40
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 206
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The participants gave similar results in regard to the tools. Similarly, the e-learners
who did not have any training on LMS used the new tools more than the trained e-
learners (Table 6.6.2-2):
Tools
Train LMS Graphs VIT VIT
MessageTag
avatars Nodes Centrality
# χ2 # χ2 # χ2 # χ2
1 Yes 11 11 11 11
2 No 26 26 26 26
3 Other 1 1 1 1
4 Missing 2 2 2 2
Total 40 40 40 40
As before, the responses are similar; the participants with no training at all used the
tools more than the others, less the ones attended an ICT course and even less the
ones who had proper training within universities. This may mean that the more
experienced the e-learners the more reluctant they are in the use of new tools.
Another explanation may be that the users who were familiar with Moodle hardly
noticed the new tools in the research pool and acted by habit.
Pearson’s correlations were also calculated for the two tables. The variables were
checked for being ordinal or categorical, no cell to have expected values less than
one, and no more than 20% of the cells have expected values less than five (Muij,
2003). In addition, because in the two rows and two columns (two-by-two tables)
case of the second table, chi-square becomes unreliable, Yates’ correction of
continuity was performed instead. Although there are arguments in the research
community against its use (e.g. Haviland, 2007), Yates’ correction of continuity is
automatically calculated in SPSS statistical output to prevent overestimation of
statistical significance for small data.
Overall, it appears that the graphs and avatars as well as MessageTag received
higher scores than VIT Nodes and Centrality. It seems that the first were found more
practical and relevant to the specific practice. In particular, the visibility of active
participation levels structure in the graphs and avatars may have helped e-learners’
reflection on their participation. Their effectiveness is a device for information
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 207
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
retrieval as the visual objects (graphs and avatars) provide users with the elements
to explore and reflect. E-learning tools need to address such participation trajectories
(Suthers, 2006). Accordingly, the visibility of the collaborative learning structure in
MessageTag seemed to help e-learners’ reflection and increased challenges per
argument. However, if compared to the results in the previous studies, for
participants lacking certain skills, training and experience in association to using new
tools may have been a disadvantage. Nonetheless, opening interaction spaces in
participation, collaborative e-learning, and social networks seemed to be essential.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 208
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The target in this study was to investigate participation in the Greek teachers’ e-
learning community and provide tools and evaluation techniques to support their
development. In the intervention analysis the form of the effect, as in level, slope, and
cyclicity, was successful. The treatment was immediate and its permanence was
continuous to this point in time. More specifically the overall discussion on the
suggested tools and evaluation techniques is as follows:
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 209
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
E-LEARNING ENGINEERING
Design Stages Ethnotechnology
1. Prerequisites: real need & right R
A INTENT attitude
2. Goals: stakeholders’ goals, Scenic E
requirements & variances fieldwork
A
3. Documentary review L
4. Background studies: description of
stakeholders background & -
characteristics Ethnographic T
5. Interdisciplinary research inputs
• Evaluation for the targeted I
B PLANNING
pedagogical approach (intentional M
variance)
E
• Evaluation for the targeted
technical problem (operational
variance)
E
• Pedagogical Usability guidelines
V
6. Design for initial design
Implications A
• Apply guidelines from for design
feedback for design L
ITERATIVE 7. Create one prototype for user-learner
C U
testing
DESIGN
• Evaluation to acquire A
feedback for redesign T
8. Redesign and evaluate with user-
learner to redesign I
EVALUATION O
D 9. Implementation in situ Scenic
& USE 10. Evaluation & feedback fieldwork N
Initially, design was used as a problem solving activity in which the problem was broken
down into a number of sub-problems, solved independently and reintegrated to produce
a design solution. In order to do this, the value of unarticulated expertise and tacit
knowledge was acknowledged. Design was found to be a reflective process having a
post hoc nature (Suchman, 1987) and evolved over time integrating design, use and
evaluation (Bannon, 1994). Such approaches have been found in the history of
engineering design as nowadays design requires flexibility between various ways of
thinking (McGarry, 2005).
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 210
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Summarise, and Other. Translating a theoretical framework into a tool was found
difficult and it was not easy to create a tool that could be used without instructions.
However, iterative design made the tool more accessible to the e-learners and it
became the most successful tool in this study.
Visualisation Interactions Tools were VIT Nodes and VIT Centrality. They depicted
the local networks in a particular discussion and were found to be less important for
collaborative e-learning. As they were built on SNA, the participants were unfamiliar
with them and it was difficult to interpret their use and incorporate it in their practice.
Another explanation may be the absence of a direct match between the task and the
method (Draper, 1993); this means that SNA methods were not explicitly connected
to the Greek teachers’ educational practices and thus, their working methods.
The Moodle discussion application was found to be rigid; in fact some participants
complained about the simplicity of the forum when they got back to Moodle@GSN.
The tools aimed at reducing misleading information and interruptions by triangulating
the otherwise separated information context and e-learners’ network; representation
of the progressive dialogue and relationships between the messages can provide a
more coherent view of the research context. This also means that e-learning
participants can manipulate information, knowledge, and interactions as interrelated
areas of one situated context in order to achieve their goals.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 211
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
It is also interesting to note that the sense of community was evident in the participants’
communication outside the course. For example, the e-tutors other than myself met for
the first time in a conference in Athens (7 October 2007). When I asked them via
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 212
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
electronic communication whether they could describe the feeling they had they said that
it was like being old schoolmates. In addition, the respondents’ comments in Other on
the final questionnaire were in accordance to the checklist. For example, P14 said he
was looking forward to login; familiarisation with the other participants and the interface
was immediate even though it was the first time he was participating in an online course.
He didn’t expect to have the same strange feeling when something like a live conference
is over. There was an interesting observation from P35 about activities and
communication: “Many people were collaborating at the same time I saw that I can
change things with my participation both as a student and as a facilitator. The formal
educational authorities do not actually know what this is about; it is a programme in
parallel with the official one (παρα-πρόγραμμα). Hope new technologies will change
things”. The participants experienced innovative collaborative e-learning tools and
techniques and stressed the importance of incorporating them in the Greek teachers’
training cycles.
Comparison with recent studies: In the recent research, Daniel in his PhD Thesis
(2007), tested 11 dimensions to sense of community within 15 undergraduate students
using an e-learning environment. There were differences in the number, age, profession
and experience of the participants; additionally, as he used a questionnaire and message
analysis to obtain his results there were several differences and similarities (p.111-126).
For example, he measured reciprocity by assessing participants’ frequency of sharing
class related resources; he related trust to the levels of awareness and co-presence. He
also measured learning as knowledge awareness depending on information about other
e-learners’ activities, that was what individual knew (competence awareness) and what
they could do (capability awareness) (p.126-127). (Note that Daniel did not always
describe his findings using numeric representation.) The similarities were:
• Participants continued their networking outside and beyond the time frame of
the class (p.111).
• The feeling of belonging to the community was high (p.113).
• There was a high ratio of shared goals and values (p.116).
• On participation and social protocols 67% of the participants said they were
aware of them.
• Peer support was evident and reinforced members’ sense of belonging
(p.120).
• There were high rates of social networking.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 213
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
• Most participants did not know each other before the course (56% said they
knew nobody, p.124-125).
• E-learners’ knowledge awareness bred trust to the community. And
• People discovered they had similar problems and interests.
To finalise, none of the participants knew the specific aims and objectives of this
project when they replied on the open question about the best things they
experienced in the course; these were: their participation in collaborative activities
(11 responses); feeling of belonging to something greater (7 responses); having
common interests and targets (6 responses); and the use of new technologies (9
responses). (For all results see A_XII_4). P2 said that she needs ‘to look for people
more, since there are others who look for the same thing and can be useful’. (Να
ερευνώ από δω και πέρα περισσότερο γιατι υπάρχουν τελικά και άλλοι που ψάχνουν
το ίδιο και μπορούν να φανούν χρήσιμοι πάντα.). P47 expressed her ‘satisfaction that
there are many colleagues interested in the further development of educational
practices’. (...ικανοποίηση ότι υπάρχουν αρκετοι συνάδελφοι που ενδιαφέρονται για
την βελτίωση εκπαιδευτικών πρακτικών.) P45 said that ‘the still waters start to move
in the area’. (...Πιστεύω ότι σιγά - σιγά αρχίζουν να κινούνται τα λιμνάζοντα ύδατα
στο χώρο.)
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 214
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
REFERENCES
Äkkinen, M. (2005). Conceptual Foundations of Online Communities (Working Paper
W-387). Helsinki: Helsinki School of Economics. Retrieved 14/02/2006, from
http://helecon3.hkkk.fi/pdf/wp/w387.pdf.
Bannon, L. J. (1994). Use, Design & Evaluation - Steps towards an Integration. In D.
Shapiro, M. Tauber & R. Traunmueller (Eds.), The Design of Computer-
Supported Cooperative Work and Groupware Systems (Vol. 12). Amsterdam,
The Netherlands: North - Holland.
Beaudoin, M. F. (2002). Learning or Lurking? Tracking the 'Invisible' Online Student.
The Internet and Higher Education, 5(2), 147-155.
Blau, P. (1964). Exchange And Power In Social Life. New York, NY: John Wiley &
Sons.
Boase, J., Horrigan, J., Wellman, B., & Rainie, L. (2006). The strength of internet ties.
Washington, DC: Pew Internet & American Life Project. Retrieved
09/05/2006, from http://www.pewinternet.org/pdfs/PIP_Internet_ties.pdf.
Bock, R. D., & Husain, S. Z. (1950). An adaptation of Holzinger's B-coefficients for
the analysis of Sciometric data. Sociometry, 13, 146-153.
Borgatti, S. P. (2005). Centrality and Network Flow. Social Networks, 27(1), 55-71.
Retrieved 23/11/2006, from
http://www.analytictech.com/borgatti/papers/centflow.pdf.
Borgatti, S. P., Everett, M. G., & Freeman, L. C. (2002). Ucinet for Windows:
Software for Social Network Analysis. Harvard, MA: Analytic Technologies.
Breiger, R. L., Boorman, S, A., & Arabie, P. (1975). An algorithm for clustering
relational data with applications to social network analysis and comparison
with multidimensional scaling. Journal of Mathematical Psychology, 12, 328-
382.
Braha, D., & Bar-Yam, Y. (2004). Information flow structure in large-scale product
development organizational networks. Journal of Information Technology, 19,
244–253.
Daniel, B. K. M. (2007). A Bayesian Belief Network Computational Model of Social
Capital in Virtual Communities. Unpublished Research, PhD Thesis,
University of Saskatchewan, Canada. Retrieved 01/08/2007, from
http://library2.usask.ca/theses/available/etd-07132007-
141903/unrestricted/ben_m.pdf.
De Nooy, W., Mrvar, A., & Batagelj, V. (2005). Exploratory Social Network Analysis
with Pajek. Retrieved 12/02/2006, from http://vlado.fmf.uni-
lj.si/pub/networks/data/esna/default.htm.
Delich, P. (2006). Pedagogical and interface modifications: What instructors change
after teaching online. Published doctoral dissertation, Pepperdine University,
Malibu, CA. Retrieved 10/09/2007, from
http://proquest.umi.com/pqdlink?Ver=1&Exp=09-09-
2012&FMT=7&DID=1144195631&RQT=309&attempt=1.
Draper, S. W. (1993). The notion of task in HCI. Paper presented at the Interchi'93, in
the Adjunct proceedings. Amsterdam, The Netherlands, April 24 – 29, 1993.
Retrieved 02/04/2006, from http://www.psy.gla.ac.uk/~steve/task.html.
Enyedy, N., & Hoadley, C. M. (2006). From dialogue to monologue and back: Middle
spaces in computer-mediated learning. International Journal of Computer
Supported Collaborative Learning, 1(4), 413-439.
Fernandes, C., & Montalvo, A. (2006). E-Quality Synthesis Report: Experience-based
Quality in European ODL.
Freeman, L. C. (1979). Centrality in Social Networks: Conceptual Clarification. Social
Networks, 1, 215-239.
Goleman, D. (2007). Social intelligence: The New Science of Human Relationships.
New York: Bantam Book.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 215
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 216
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 217
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
Conclusions:
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 218
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
7.1 INTRODUCTION
The aim of this chapter is to examine the scope of the findings, to put them into
the context, scrutinize the strengths and limitations of the work and suggest
implications for findings and directions for the future investigations might take.
In this section the scope of the research and the overall findings are described
in relation to the aim and objectives. This study aimed to carry out the following with
regard to Collaborative e-Learning Communities:
Thus, this study set out to answer a number of exploratory questions; the conclusions
are presented next.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 219
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
last step towards community knowledge building and thus, resulting in the e-learning
passive mode. In other words, available learning management systems do not
support users’ in co-creating the e-learners-generated context.
There have not been many attempts to support collaborative e-learning from a
community viewpoint; finding suitable variables to describe and support increasing
participation was part of the problem. Additionally, design was found to be
inadequate to carry the social interactions; it is legitimate for the participant to be in
any participation level. As a result, all Greek teachers never crossed the threshold of
passive participation for 3 years because of personal, social, learning, institutional or
technical obstacles. In other words, if the e-learners were adequately supported by
tools and evaluation techniques then passive participation would drop to the
minimum.
For the past 50 years two main trends have been observed in education, the socio-
cultural focus and the use of technology. However, these two trends have evolved
almost separately. Socio-technical design and user-centred design were planning
approaches aiming to acknowledge that the development of interactive technologies
increasingly relies on an appreciation of the social circumstances in which systems
are used. Design is planning; educational or instructional design is the systematic
processing of activities to solve an instructional problem with the aid of technologies.
However, educational design and in particular e-learning design neglected the dual
and situated persona of the learner; she acts as both a user and a learner. In
addition, the e-learning systems were found to be information-based mainly
supporting monologue instead of being communication-based towards dialogue. For
this reason they fail to support e-learners’ transition between internalisation to
externalisation and becoming active participants. Thus, mere provision of information
points to poor e-learning quality. So, if educational design could understand the
technology of collaborative practice, e-learning quality could be improved.
The first signpost was the radical view that design is planning with a post hoc nature.
This means that situated design needs to follow the evolution of its context, in this
case, the collaborative e-learning community. The second signpost was the
development of a social and technical infrastructure that supports the key activity, in
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 220
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
this study, collaborative e-learning. The third signpost was the real-time setting of the
key activity and its evaluation. The last signpost was the involvement of e-learning
participants in the design. These guidelines suggested ethnotechnology as the
methodology to inform design on a social, learning, and technical level.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 221
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
7.3 CONTRIBUTIONS
The post hoc structure for E-Learning Engineering was revealed integrating design,
evaluation, and use, this involves all e-learning participants in design, and is informed
by ethnotechnology and real-time evaluation tools.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 222
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
The participation graphs and avatars that depicted active participation were found
to have supported increasing participation in collaborative e-learning. This was due to
the fact that they were built on the technology of participants’ real practice.
MessageTag was the tool built on the CeLE analytical framework. As with the
participation tools, it successfully supported e-learners because of its relevance to
the key activity, that was collaborative e-learning.
The Visualisation Interactions Tools Nodes and Centrality were found to be less
supportive in the e-learner’s endeavour and had the most problems from a
pedagogical usability viewpoint. However, it appears that their use aided the
participation of the Greek teachers in their e-learning community.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 223
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 224
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
From a design viewpoint, the tools were not implemented in the real e-learning
environment for the Greek teachers future use. In addition, I had too many roles and
limited programming skills resulting in not being able to tackle a problem on the log
files for three of the tools, on time. Also, there was no usability testing of the tools in a
usability laboratory.
Lastly, the suggested tools and evaluation techniques need to be further tested and
developed in different contexts to ensure their validity and reliability.
7.5 CONCLUSIONS
This final section presents the overall blocking factors responsible for the
Greek teachers’ passive participation, the implications from the findings, and future
research directions.
The problem of e-learning quality was found to be related to the Greek teachers’
passive participation. A network of institutional, instructional, technical, and personal
obstacles acted as blocking factors on a micro and macro level:
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 225
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
In fact, these blocking factors have been reported on a European level in the recent
E-quality report (Fernandes & Montalvo, 2006) and were also identified in my
participant observation in 2 European e-learning projects. Thus, the problem of e-
learning quality in the Greek School Network was not different or more intensive than
in other European countries. This also means that although this study was designed
to be strictly situated, the recommendations for researchers, e-learning engineers, e-
learning practitioners, and the Greek educational authorities may have an impact on
e-learning quality on a broader level (see recommendations in Appendix XIII).
The findings in this study demonstrate the significance of social, learning, and
technical aspects of e-learning. Two interventions were made, collaborative e-
learning and the introduction of new tools; the first was related to community
management and collaborative e-learning, the tools were built to facilitate them. The
conceptual frameworks of the participation levels, the Sense of e-Learning
Community Index (SeLCI), the Collaborative e-Learning Episode (CeLE) analytical
framework, and the associated tools aimed to bridge the social and learning gap.
Brief description of the findings, their implications and recent related studies will be
discussed next.
Within the context of the study, the participants’ prior knowledge and ability to interact
was enhanced (Yang, 2002); there was also lack of governmental planning for e-
learning pedagogy, technology, and legislation. The implications from the first finding
suggest the need for training the Greek teachers in current pedagogical collaborative
approaches and new technologies; improvement of soft skills and communication
techniques; development of shared vision and goals; access to professional help and
support. These implications point to the problem of e-readiness for e-learning (e.g.
Kaminsky & Currie, 2008). The failure for e-readiness is evident in the great
discrepancy between different ranks in Greece (The Economist Intelligence Unit &
IMB Corporation, 2003:16); on a scale from 1 to 10, 8.80 was the government rank,
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 226
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
5.87 the industry rank, and 5.66 the society rank. Thus, the implications is that
organisations and business need to understand and correctly judge real situations,
trace solutions as processes, and educate and support everyone involved in e-
learning.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 227
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
The development of collaborative learning networks and shared practice has been
found to be themes for continuing professional development (e.g. Pickering et al.,
2007). First, communication and problem-solving skills were improved by the e-
learners’ interactions. They were evident in the more complex discussions at the end
of the course comparing to monologues as redundant messages at the beginning of
the course. The analytical framework of collaborative e-learning episodes (CeLE) and
the associated tool MessageTag managed to bridge methods and tasks and provide
simple to use and reliable assessment of the e-learning quality. An implication is that
increasing reciprocity can facilitate the transition from monologues to dialogues as
there are increasing clues as opportunities for critical engagement in dialogue. A
second implication is related to the e-tutors’ ability to guide the e-learners into the
journey of critical thinking and knowledge co-construction and then leave them on
their own capabilities.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 228
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
The use of new tools was found to support collaborative e-learning to an adequate
extent. The implications are related to the need for numerical and graphical
evaluation tools that demonstrate their purpose of use. Such tools can increase e-
learners’ focus, intensity, and persistence in social and learning interactions. E-tutors
can direct and control online discussions, discover e-learners’ weaknesses and
strengths, activate the lurkers with specific questions, aid in team building by
“bonding” the team, and record the discussion. From a design viewpoint, several
groups need to evaluate learning technologies and applications anchored in their
different personas such a novice and expert users (Faulkner, 2000) as e-learners as
well as developers and e-tutors. A pedagogical usability evaluation framework can be
used to evaluate usability and utility of new tools. Designers need to seriously
consider the social aspects of learning, formative and summative evaluation, and
facilitate e-learners increase control as in self-emerged collaborative learning (Dron,
2007; Anderson, 2007). Such tools need to function on multiple levels supporting
interaction for individuals, small groups and networks or as Anderson (2008)
suggested, employ social software. This implication depicts the limitations of current
interactive applications; in the recent Communication of the ACM journal, Hendler
and colleagues stress that ‘today’s applications are very early social machines,
limited by the fact that they are largely isolated from another’ (2008:65).
The interventions built the types of communities of practice discussed by Kanes and
Lerman (2008). This is because the social and learning aspect was anchored in
legitimate peripheral participation as in Community of Practice Type 1 (CPT1).
However, it created an opposing dynamic in the Greek educational system by
suggesting that the community needs to move forward and acquire the new
competencies and technologies for the 21st century. This creates a conflict within the
educational system as with Community of Practice Type 2 (CPT2) (Kanes & Lerman,
2008): ‘CPT2 is built around tension, conflict and discontinuity of practice and
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 229
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
A final implication for research was connected to triangulating the data via
quantitative, qualitative and social network analysis; 3 different research perspectives
can open possibilities beyond initial propositions and hypotheses. Overall, there is
room for research on usable, useful and reliable tools and evaluation techniques to
assess different levels and types of participation and critical thinking in
collaborative e-learning.
• Research
o Multidisciplinary research on the social, learning, and technical
aspects of e-learning activity.
o Ethnotechnology as a methodology to advance community members’
own practices.
• E-learning readiness
o Institutions and organisations assessment for e-readiness on a
technical and pedagogical level.
o Soft skills as part of teachers’ training and professional development.
o Resistance to change as a longitude study of e-learning communities.
• E-Learning
o E-learning management as a social as well as a learning process.
o The process of e-learning and the ways different learning styles
influence it.
o The e-learning styles in relation to social intelligence.
o The impact of the relationships between the Collaborative e-Learning
Episode’s attributes.
o Design for each of the Collaborative e-Learning Episode’s attributes.
o Design for learning objects and their re-contextualisation based on the
different critical thinking levels in Collaborative e-Learning Episodes.
o The relationships between personalised and collaborative e-learning
and their impact to e-learning quality.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 230
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
• E-Learning Community
o Community design, management, evaluation and development, and
the attributes that most facilitate e-learning to occur.
o The impact of the relationships between the Collaborative e-Learning
Episode’s attributes.
o Design for each of the Sense of e-Learning Community Index’s
attributes.
o Evaluation of participation and its impact on students’ performance.
o The impact of interactions in e-learning across a learning network,
within groups and on an individual level.
o The role of decision making and leadership in e-learning communities.
• Design
o Multimodal interaction and interoperability between educational
technologies.
o Design to support personalised and community e-learning
environments.
o Design to support community management and evaluation.
o Tools based on numeric and graphical representation for formative
and summative evaluation.
o The role of pedagogical usability in facilitating e-learning.
o Specific pedagogical frameworks and design on associated tools.
The discovery of the mirror neurons as the neurons that enable the representation of
other humans in the brain, social intelligence, current e-learning research, the new
communication technologies and social software, as well as this study, indicate that
collaboration is part of the human nature. Building on collaborative e-learning
communities can ensure e-learning quality; collaboration is written into our DNA.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 231
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
REFERENCES
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 232
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 7: Conclusions
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 233
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
1. Information.
− Informative first page: Ensure that the community’s first page provides all
needed information to encourage and inspire the students to engage.
− Introduction of themes in collaboration with the students and the Students
Administration Office.
2. Registration system: Facilitate the registration process if necessary.
3. Welcome note: A welcome note inspires and encourages the students to
participate as well as giving additional information on technical and management
issues (e.g. software description, inappropriate behaviour etc.)
4. Profiles: The students need to be encouraged to construct their profiles. Profiles
provide a feeling of co-presence and enhance the sense of belonging.
5. Induction and training: An initial meeting for using Moodle might bring issues of
usability of the system and suggest the problems students have on using the system
on site (if any). Additionally, information will be provided on ways for writing, replying
and form an online message.
6. Subgroups: Based on students’ research interests as well as hobbies sub-groups
will create initial locus of interactions.
7. Initial one-way communication: Introduction of the students (research interests,
hobbies, personal information that would like to share, experiences etc) as well as
tools (e.g. votes, polls, surveys, newsletters) can break the ice and give the
necessary information to move to two ways of communication and productive
interactions.
8. First message: The first of each student might define later behaviour. An initial
authentic reply and warm welcoming will encourage students to continue
communication as well as shy students to send a message.
9. Discussion highlights as newsletters: A monthly newsletter that will provide a
summary of the discussions and any additional information. Newsletters inform
members for news, activities, make members aware of the previous issues and
develop a felling of belonging to a community.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 234
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Twenty five (25) messages (N=47, 53,1%) appeared to have a pattern: an initial
introduction as a response to the selected message, an extensive explanation and
justification of their point was made, an example was making suggestions very clear
and lastly, a greeting or an interesting quote used to ‘sign’ the message. A detailed
description is following:
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 235
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
The British Institute of Risk Management (IRM), the Association of Insurance and
Risk Managers (AIRMIC) and the National Forum for Risk Management in the Public
Sector (ALARM) (2002) have published a standard risk management procedure. Risk
can be defined as the combination of the probability of an event and its
consequences (ISO/IEC Guide 73). Risk assessment found to be important for this
study after the initial findings on the absence of cooperation with the Greek
educational authorities as well as the absence of the Greek teachers’ participation in
collaborative activities. Risk management increases the probability of success and
also reduces the probability of failure and uncertainty. Even though it is usually a
continuous and developing process, it was only used for the specific purpose of
completing the course under the desirable circumstances. In other words, a shift to a
different research context was not desirable.
Risk Description
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 236
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Based on the risk description, it appears that in the first case of implementing the
tools in the real e-learning environment developed for the Greek teachers the risk is
very high. Despite the fact that the Deputy Director of the Greek School Network
Technical Support said that the tools will be implemented, the risk estimation is 95%.
(A high risk is usually more than 25%). This is because cooperation with the Greek
educational authorities found to be 0% on more than one occasion. However, in the
second occasion of supporting Greek teachers in e-learning the risk is very low, less
that 2%. This is because the chance of enabling participation is considered to be
more than 75%.
The above description and risk estimation suggest that a shift to the study from a real
implementation to a quasi experimental one cannot completely alter the results and
significantly affect reliability and validity. In other words, the Hawthorn effect is
always a threat in research either in real or experimental environments. Having a
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 237
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
linked research pool can be the implemented measure to tackle the first risk;
provision of adequate help and support based on initial instructional learning and
then shift to collaborative e-learning can be the risk treatment for the second risk of
absence of participation.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 238
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 239
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 240
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Your contribution to this research is more valuable. The information you will provide
will provide a process for the Greek teachers’ active participation in the online
community and will reduce the time of engagement. In addition, your opinions and
propositions will present existing e-learning problems. The information you will
provide are strictly confidential and will only be used for research purposes. The
results from this study will be available to you after its completion (September 2007).
Despite the fact that the questionnaire is rather long, there was an effort to be easy to
answer, with accuracy and speed in 30 minutes. It is consisted of the following 5
sections:
The questionnaires should be sent to Niki Lambropoulos until the 15th of April 2007
the latest, at the email nikilambropoulos@gmail.com
You can use this text box to write your name and address in Greece in order to send
you the certificates of participation from London South Bank University and the
Greek School Network. They will be sent after all questionnaires are sent, late April
2007 the latest.
Best regards,
Niki Lambropoulos
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 241
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
FINAL QUESTIONNAIRE
You are invited to dedicate some of your precious time to fill in the following questionnaire. You can
fill it in the way you prefer as there are not right and wrong answers –only different pinions-; therefore
some questions are not very clear. The questionnaire was based on Word Processor form function; you
simply click in the grey areas and select one of the answers (unless stated otherwise) on a scale from 1
(low) to 5 (very much) or simply write. The selection Don’t know / Don’t answer is referred as N/A.
1). How many people did you know in person before the online course?
2). Do you think there was community development in time on the online
course?
2ai. If you think community development was evident, can you describe some
elements that prove it?
2bi. How did the discussion forums help/restrict the development of a sense of
belonging to the e-learning community?
3). Did you develop any relationships, either friendly or professional with
the other co-learners?
3ai. If yes, do you think they will continue outside the online course?
4). In your own opinion, what was the element that hold the community
members together, if any?
5). In your own opinion, most of the participants shared their values.
6). In your own opinion, most of the participants had common interests
and goals.
(a) Yes (b) No (c) N/A
7). How much do you think that the members were helping each other in
the online course?
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 242
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
(a) Very (b) Not much (c) A little (d) Almost no help
(e) N/A
8). How long (in days) do you think it took for the community to emerge
– develop a sense of working together?
11ii. Somebody has to be very careful in the online course because some
participants wanted to take advantage of people and situations.
1 2 3 4 5
In a scale from 1-5, where 1 means
12). very little and 5 a lot, how much do Very Small Great Very
Neit
you thin the participants trusted Small exten exten great
her
each other in the online course? extend d d extend
Individuals who had similar
a
specialisation
Individuals who had similar writing
b
skills
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 243
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
c E-tutors
Individuals who seemed to have
d
advanced knowledge on the subject
e You cant trust anyone
13). In general, do you think that the level of trust during the online
course:
(a) Increased (b) Decreased (c) Remained the same (d) N/A
15). Do you think that YOUR participation was useful to the e-learning
community?
(a) Yes (b) No (c) N/A
16. After your registration to the online course, from whom did you learn issues
that were of your interest?
16a After your registration to the online course, how did you learn issues that
were of your interest? Can you describe some ways?
17). Do you thin that there were roles developed between the members?
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 244
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
19. Do you think that the members who DID NOT participate actively had to
remain in the course?
20). Do you think that new members in the e-learning community can contribute
to existing knowledge?
21). Do you think that cooperation with other professional communities (e.g.
programmers, multimedia developers, etc) is necessary?
22). The participants were specialised in different areas. Do you think that this
helped collaboration in the online course?
(a) Very (b) Not much (c) A little (d) No (e) N/A
23). Did you learn ways to enhance collaboration using new tools in the
online course?
24). What were the most important examples of collaborative learning for
you?
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 245
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
25). Who did you MOSTLY turn for help when you faced some
problems?
Did you think that the tool design was functional (easy
30 to use)?
1 2 3 4 5
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 246
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Did you think that learning how to use the tool is time
33 consuming and tiring?
1 2 3 4 5
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 247
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
1) Ε. PROFESSIONAL DEVELOPMENT
WITHIN CoP
41. Which one of the statements describes the common space between discussions
and your own educational practice? Please choose accordingly (more than one
answers are allowed):
(f) Other
44. How many times have the educational authorities from the Greek Ministry of
Education contacted you to ask your opinion on learning technology issues?
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 248
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
(a) Many times (b) Some times (c) A couple of times (d) Never
45. How many times have the educational authorities from the Greek
Pedagogical institute contacted you to ask your opinion on learning technology
issues?
(a) Many times (b) Some times (c) A couple of times (d) Never
You can use this space for more comments and observations:
Name (Optional)
Niki Lambropoulos
Research Student,
Centre for Interactive Systems Engineering
Faculty of Business, Computing and Information Management
London South Bank University, London, United Kingdom
http://www.lsbu.ac.uk/bcim/research/cise/ * http://nikilambropoulos.org &
1 Dale Grove London N12 8EE U.K.
Tel: +44(0)2084465909
Email: nikilambropoulos@gmail.com * niki@intelligenesis.homechoice.co.uk
Skype: niki.lambropoulos
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 249
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Only samples from the participants’ documents are presented here; these were the
following:
A_VI_1. Invitation to the study
A_VI_2. Netiquette
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 250
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 251
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Demographics
Variables Values # Participants Percent
20-30 7 22.6
30-40 10 32.3
1 Age
40-65 14 45.2
Missing 0 0
Female 12 39
2 Gender Male 19 61
Missing 0 0
1-5 11 35.5
6-10 5 16.1
Working
3 11-20 11 35.5
Experience
40+ 4 12.9
Missing 0 0
1-5 4 12.9
6-10 14 45.2
Use of computer
4 11-20 10 32.3
- software
20+ 3 9.7
Missing 0 0
No use 1 3.2
1 -2 h 17 54.8
Use of new
3-5 h 6 19.4
5 technologies in
6-12 h 4 12.9
class: h/w
12+ h 3 9.7
Missing 0 0
No use 0 0
1 -2 h 13 41.9
Use of new 3-5 h 8 25.8
6 technologies for
6-12 h 5 16.1
education: h/w
12+ h 5 16.1
Missing 0 0
No use 1 3.2
1 -2 h 21 67.7
Use of Internet 3-5 h 4 12.9
7
in class: h/w 6-12 h 2 6.5
12+ h 3 9.7
Missing 0 0
No use 0 0
1 -2 h 8 25.8
Use of Internet
3-5 h 8 25.8
8 for education:
6-12 h 11 35.5
h/w
12+ h 4 12.9
Missing 0 0
0 12 38.7
1-5 16 51.6
9 Use of LMS
6-10 2 6.5
Missing 1 3.2
Simple 7 22.6
OK 19 61.3
Moodle Difficult 2 6.5
10 N/A 2 6.5
(std.d: 0.92)
Other 1 3.2
Missing 0 0
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 252
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
E-Learners' perseptions on
Moodle usability level
2, 7%
2, 7% 7, 24%
Easy
OK
Difficult
Don't Know
Missing: 1
18, 62%
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 253
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 254
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 255
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 256
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 257
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Introduction
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 258
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Participant SP1 started with a question as the title of the discussion (AIa-1:stanza3).
Discussion Title: How we can create a website in two or more languages?
Participant SP1 elaborated the subject and indicated the aim of the discussion (AIa-
1:stanza4).
I want to make a website in Greek and English (or and more) Languages.
Response 1 (AK1).
The first was an explanation with an example (AIa-1:stanza14).
The only way I know is having a template and then editing-translating the content and the
buttons. This site (website) has great templates, but you ought to have some experience with
Photoshop to edit them.
Then there was a reference to AK1’s experience and evaluation took place by
referring to best practice and justification of the evaluation (AIa-1:stanza14).
I prefer this way, because I don't have to code or design my webpages from scratch.
A conflict stressed a comparison and initiated justification for best practice (AIa-
1:stanza15).
but I'm a little bit lazy and want to get the best results with little effort, so I prefer Photoshop
file format
Participant AK1 closes his message with an emoticon declaring satisfaction (social
cue) (AIa-1:stanza15).
Response 2 (SD1).
The answer from Participant AK1 was agreement based on the same interest and
provision of more information as the initiation of his message (AIa-2:stanza18).
This is a very interesting question, now we cooperate across countires, especially working on
EU educational projects. Sometimes international websites contain a cookie that will search
and detect the regional settings your computer is using and they will present the version that
is compatible with those settings, including the country (regional settings)….
An alternative solution initiated the aim of the next verse by replying explicitly to SP1
(you) (AIa-2:stanza20)
If, however, you want to design a webisite can be both Greek and English…
Based on the previous Participant’s SP1 message explores the issue further (AIa-
2:stanza20).
tehn you will ahve to resort the translation of teh website and use,…
Participant SD1 agrees with a previous message and refers to the name of the
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 259
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
SD1 justifies and evaluates the proposition with a comparison referring to practical
implementation (AIa-2:stanza20).
This will make your job easire.
SD1 closes the message by providing more information as well as evaluating the
given information (AIa-2:stanzas22-24)
More on CSS can be found here>
http://www.w3.org/Style/CSS/ this is basic
and some more http://www.developertutorials.com/css-2/cover.html
Response 3 (NL1)
The Researcher replied by exploring the issue further, explaining her solution on a
project (AIa-3:stanzas27-28). The researcher tried to get involve in the discussions
as minimum as possible.
A different approach is the bilingual one.
I built the Greek School of London website here <http://dim-lon.europe.sch.gr/> If does not
work > (pic available)
Response 4 (DD1)
DD1 started his message by greeting the person who sent the initial question (social
cue) (AIa-3:stanza30).
Hi, S.
He agreed with a previous message and provided his own individual solution (AIa-
3:stanza31).
Yes you can make a website in two or more languages. A good example is my personal home
page (<http://users.ker.sch.gr/geoker>)
After exploring and explaining his suggestions, DD1 made an evaluation on a topic
as regards the design (AIa-3:stanza33).
It is best to give names to your files which facilitate grouping
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 260
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
The he finished his message with a greeting and providing his name (social cues)
(AIa-3:stanzas35-36).
Cheers. D.
Response 5 (MF1)
MF1 adopted DD1’s style of structuring the message. The message has a greeting
as an initiation; MF1 refers to a person from a previous message and ends the
message in the same way, indicating empathic feelings reflected in her writing.
MF1 started her message by greeting everyone (social cue) (AIa-3:stanza38).
Hi everybody
Then she agreed with DD1 referring to his name (social cue), the participant who
sent the previous message (AIa-3:stanza39).
I agree with D.
Imitating DD1, MF1 she finished her message with a greeting and providing her
name (social cues) (AIa-3:stanzas45-46).
cheers
M
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 261
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
AIM – Q1
Q2
(AK1 ) explanation
Example
Evaluation (AK1 experience)
Justification (reasons)
Further Information
Conflict to justify
Aim
Evaluation
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 262
Tools & Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 263
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 264
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Problem solving 1
Learn something interesting 1
Learn something useful on
1
a personal level
Learn something useful on
1
a community level
Implementation of new
1
experience
Implementation of new
1
knowledge
Participation in planning 1
Presentation of own
3 Discussions 1
experience/work
Opinions 3 Ideas 1
Questions 2 Opinions 1
Information 2
Propositions 2
Knowledge Discussions 2
Exploration 1
Comments 1
Advice 1
Suggestions 1
Answers 1
Use new tools 1
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 265
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 266
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 267
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
CeLE-III was completed in one day (05/03/2007) and described the stages with
which P18 found the solution to a registration, problem based on her colleagues’
suggestions (CeLE-CIII: Stanzas 4-23). (Numbers on stanzas occurred automatically
in Atlas-ti™.)
> P18 initiated the discussion with a statement about the exercise they had, that was
creating a blog (stanza 5): I have created a blog in Pathfinder
Then P18 describes her aim (stanza 5): but I would like to comment on Blogger.
P18 refers to the problem (stanza 5): but every time I log out [the system] it does not
recognise the login name and password.
> P24 asked P18 a question to clarify the problem (stanza 8): Do you add
yourname@gmail.com for username?
> E-tutor P32 explained what he had to do (stanza11): you simply write the user name
you have from gmail
> Then, R provided an example based on P24 and P32’s suggestion (stanza 14): e.g.
login name: nikilambropoulso@gmail.com password:
nbn3vb4325rb43wqbrliqwyfiuQGFLCBAf
> E-tutor P52 asked more clarifying questions (stanza 17-18): Which blog do you want
to use to comment? Are you allowed to post as administrator? Are the login name and
password incorrect?
> Then P52 asked for a following up on the process (stanza 18): Keep us updated on
the process…
> He finished his message with a social cue (stanza 18): pleaaaaaaaaase!
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 268
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
> P18 replied with a social cue (stanza 20): Thank you all for your help.
> He found what the problem was (stanza 20): I found what was wrong with it.
He explained what the problem was (stanza 20-22): When I first got the login name and
password I wrote the mail address, not the user name and this resulted the system to ask for
this address as a user name.
P52 sent greetings and thanked for the discussion (stanza 23): be well, thank you for
the message.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 269
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 270
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
CeLE-III is relatively small (133 words) with more information on initiation and social
cues. The agreement and conflict stage was missed; three exploration levels were
built on participants’ questions. The following list represents the discussion in a linear
form as a collaborative problem solving process:
Initial post (P18): Statement – Aim – Problem – Question – Social cue (Help)
(P24 ) Exploration
(P32) Exploration: suggestion
(R) Explanation: example
(P52) Exploration – Initiation
CeLE-IX was completed in 3 days (29-31/03/2007) and referred to the use of specific
e-learning tools in the project creation process. The discussion was initiated outside
the online course between P37 and P22; then the participants shared their
proposition with the rest of the group (CeLE-GIX: Stanzas 2-84). (Numbers on
stanzas occurred automatically in Atlas-ti™.)
> P37 initiated the discussion by presenting a proposition worked and agreed with
P22 (social cue) (stanzas 2-9): After a conversation with P22 about the way e-learning
tools can be implemented in the project method stages, we propose:
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 271
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
[The attached ppt image described an example using Web 2.0 and Web 1.0 tools:
Free subject selection and initial design was related to videoconferencing;
Research – information search and Elaboration was related to a blog;
Synthesis – production was related to a wiki;
Presentation was related to videoconferencing and a wiki; and
Evaluation was related to videoconferencing and a wiki.]
> O2 was an e-tutor. He replied the same day and tagged the message as INFORM
(29/03/2007, stanzas 10-16). His points were directly disagreeing with P37 (stanza
12): I disagree with ‘Research – information search and Elaboration’ and with
‘Presentation’ as regards the use of tools.
She agreed (stanza 21): But this is exactly what is going to happen in a blog,
She explained the reasons for this decision with an example and stressed her
opinion by highlighting and bolding the main points of her argument (stanza 21):
recording and commenting of the data gathered and focused on events and situations for
their elaboration. Obviously, I don’t do the search via a blog.
Then she explored the topic further and provided alternative points using bullet points
(stanzas 22-27): The presentation in the World Wide Web using a wiki was [suggested] for
the following reasons:
The whole tam will create the context and its contents
The whole team will follow the stages before the final product
The whole team will evaluate [the project] and will be evaluated
And lastly it is recommended for navigation via the links, whereas the blog is somehow still –
unless the presentation is very short.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 272
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
> After 4 and half hours, she made a new proposition as continuing co-construction
based on previous arguments (30/03/2007, stanzas 28-37): [This is] The advanced
diagramme after the dialogue with O2:
The blog is used only for data recording and commenting as regards events and situations
gathered for further development. Search and data collection is conducted by the usual
methods: Web, literature review, educational visits, discussions with specialists etc.
In addition, the presentation can be done in a web-site, if we want to have more multi-media
elements.
[The attached ppt image described an example using Web 2.0 and Web 1.0 tools:
Free subject selection and initial design was related to videoconferencing;
Research – information search and Elaboration was related to WWW, literature
review and a blog;
Synthesis – production was related to a wiki;
Presentation was related to videoconferencing, a wiki and a website; and
Evaluation was related to videoconferencing and a blog.]
He signed off with his name (social cue) (stanza 43): P50
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 273
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
> (The following message was not on the logical order for the argument as reply to
messages can be interrupted in discussion forums.) P58 tagged his message as
QUESTION (30/03/2007, stanzas 48-50).
He asked a question for all to agree (stanza 48): Do you agree that it is difficult to have
videoconferencing on the first stage if we refer to a technologically illiterate audience?
He explained his point with an example (stanza 49): Even this online course had to have
videoconferencing in the third week…
Lastly, he signed off with his initials (social cue) (stanza 50): P58
He explored P37’s argument referring to his name (social cue) (stanza 56): P37, it is
very difficult to start the first stage with videoconferencing.
This has been tested fro many years in Comenius and I think recently in etwinning with great
success. Have a look on http://partbase.eupro.se/frameuk.htm as well.
After this can be proceeded then videoconferencing can be part of the design if needed.
He agreed with the previous message (stanza 56): As for the rest of the stages I agree.
He evaluated P37’s work referring to his name (social cue) (stanza 56): Anyhow, this
was a great job, P37.
Lastly he finished the message with his first name (social cue) (stanza 58): P13.
> P37 tagged his message as EXPLORE and wrote its Greek translation in capital
letters (ΔΙΕΡΕΥΝΗΣΗ) (30/03/2007, stanzas 59-74).
He agreed with the previous messages (stanza 62): I agree with all previous speakers.
He explored the argument further and provided and example based on individual
experience (stanza 63): I just need to stress two parameters:
1) Our students are too familiarised with the new technologies. They wont face any problem
at all with videoconferencing; on the contrary, this environment is very attractive for them.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 274
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
He evaluated and agreed with P37’s work (stanza 68): Very good job P37. As you wrote
in your magic phrase ‘it is just a proposition’. It is accepted and implemented if the
situations, the teachers and the technology allow it.
He summarised (stanza 69): We learned what Blogs and wikis are and how to use them.
He thanked (social cue) P37 in advance (stanza 72): Since you created this wonderful
presentation, could you send it to me via email?? Thank you.
He finished the message with his full name (social cue) (stanza 74): P14
> P22 replied the next day and tagged his message as EXPLAIN. (It appeared that
he was defensive in his message.) (31/03/2007, stanzas 75-84).
He quoted the first part of P37’s message and stressed the fact that it was an initial
idea by changing the size of the font as well as making it bold (stanza 79): After a
conversation with P22 about the way e-learning tools can be implemented in the project
method stages, we propose:
He agreed with P37 (social cue) (stanza 80):
- At is appears in P37’s first post, this is just a proposition based on the tools we used in the
online course, including skype and msn, tools that most of our students can easily use, and in
many occasions, better than many of our colleagues.
- On the question which tools somebody will use to design, develop and complete his work,
this depends on the situation (students’ age, school, infrastructure, level, students’ mood,
etc)
- The order we propose with the tools and its implementation it is not, and it can’t be in any
way, neither strict nor unique. Each one said [that] it depends on what, how, etc.
- The positive thing is that the previous slide created an argument that lead to a creative
dialogue, and this is the essence of the online course, it is not that insignificant that a month
ago most of us did not know or heard about these tools, we hardly knew them and for sure we
had never used them.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 275
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
(P58) Question
(P13) Agreement – Initiation – Explanation - New proposition / idea –
Evaluation – Social cue
(P37) Explore
(P14) Evaluation – Agreement – Summarise –
Exploration - New proposition / idea – Social
cue
(P22) Agreement – Exploration
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 276
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 277
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
Negative evaluation 1
Best practice & justification 4
Result 1
Ideas, Co-
6 New idea 4 7
construction
Summary 2
7 Other 1 Emphasis (colour, bold etc) 15 15
Tagged
TOTAL 76/7
9.2%
∗The number in parenthesis refer to a reply to a specific person indicated by her name
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 278
Tools and Evaluation Techniques for Collaborative E-Learning Communities Chapter 6: Main Study
It is interesting to note that the interlocutors were different individuals except P37
who appears to be on the medium activity level. The participation in the discussion
appeared as follows:
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 279
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 280
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
per participant
Messages Density 2a/N(N-1) 2X868/95(95-1)=0.19
* My initiations were included as some participants’ messages were produced from these initiations.
The tables present the first three quantitative variables, richness of text, depth of discussions, and messages density. (My posts are in
parentheses.) These tables describe the results in Moodle@GSN and the research pool. The first column describes the online course section, the
second column the forums and the dates these forums were introduced and completed. Then the replies are divided into initiations found in logs
as “add forum”, and the actual replies, found in logs as “add posts”. From these messages, the sub-columns present the total number of
messages, the messages for analysis as well as participants’ only messages. The last two columns show the overall number of messages
provided the initiations for analysis including my messages, and the participants’ messages, and the richness of text.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 281
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 282
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 283
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
VIT 5 6 16 8 5
Centrality 12.5 15 40 20 12.5 3 1.1 0
Graphs & 1 2 18 13 5
Avatars 2.5 5 45 32.5 12.5 3.5 0.8 1 3.3875
MessageTag 1 2.5 1 2.5 15 37.5 17 42.5 6 15 3.65 0.8 0
VIT Nodes 5 12.5 4 10 15 37.5 9 22.5 7 17.5 3.2 1.2 0
VIT 5 4 15 9 7
11 VD: Graphics Centrality 12.5 10 37.5 22.5 17.5 3.2 1.2 0
Graphs & 1 5 14 14 6 0.9
Avatars 2.5 12.5 35 35 15 3.4 3.0875
MessageTag 1 2.5 2 5 16 40 13 32.5 7 17.5 3.65 1 1
VIT Nodes 5 12.5 8 20 13 32.5 10 25 4 10 3 1.1 0
VD: VIT 6 7 13 10 4 1.2
12 Attractiveness Centrality 15 17.5 32.5 25 10 2.3 0
Graphs & 19 14 4 3 0.9
Avatars 47.5 35 10 7.5 1.8 0 1.85
MessageTag 22 55 12 30 3 7.5 3 7.5 1.7 0.9 0
VIT Nodes 18 45 13 32.5 4 10 3 7.5 2 5 2 1.1 0
VIT 19 12 4 3 2 1.1
13 Tool failure Centrality 47.5 30 10 7.5 5 1.9 0
Graphs & 2 12 16 10 0.9
Avatars 5 30 40 25 3.8 0 3.55
MessageTag 4 10 9 22.5 16 40 11 27.5 3.8 1.1 0
VIT Nodes 6 15 1 2.5 15 37.5 12 30 6 15 3.3 1.2 0
VIT 6 1 15 12 6 1.2
14 Fast response Centrality 15 2.5 37.5 30 15 3.3 0
Graphs & 1 2 13 16 8 0.9
Avatars 2.5 5 32.5 40 20 3.7 0 3.375
MessageTag 1 2.5 3 7.5 15 37.5 12 30 9 22.5 3.6 1 0
VIT Nodes 6 15 3 7.5 17 42.5 8 20 6 15 3.1 1.2 0
Overall VIT 6 4 15 9 6 1.2
15 satisfaction Centrality 15 10 37.5 22.5 15 3.1 0
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 284
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 285
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 286
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Negative Correlations
1714 Satisfaction VIT Nodes Failure VIT Centrality -0.600
1715 Learnability MessageTag VD: Graphics VIT Nodes -0.603
1716 Accessibility VIT Nodes VD: Graphics MessageTag -0.603
1717 Support CeL Graphs/avatars Functionality VIT Nodes -0.605
1718 Satisfaction VIT Nodes Info overload MessageTag -0.605
1719 Satisfaction VIT Nodes Info overload Graphs/avatars -0.607
1720 Motivation VIT Centrality Info overload MessageTag -0.608
1721 Satisfaction VIT Nodes Failure VIT Nodes -0.609
1722 Motivation VIT Nodes VD: Graphics MessageTag -0.611
1723 Satisfaction VIT Centrality Accessibility Graphs/avatars -0.612
1724 Instructions VIT Centrality Frequency Graphs/avatars -0.619
1725 Originality VIT Nodes VD: Graphics MessageTag -0.620
1726 Motivation VIT Nodes Info overload MessageTag -0.620
1727 Satisfaction Graphs/avatars Functionality VIT Centrality -0.621
1728 Education goals MessageTag Accessibility VIT Nodes -0.621
1729 Education goals MessageTag Accessibility VIT Centrality -0.624
1730 Education goals Graphs/avatars Instructions VIT Nodes -0.624
1731 Satisfaction VIT Nodes Accessibility Graphs/avatars -0.628
1732 Motivation VIT Centrality Info overload VIT Nodes -0.628
1733 Motivation VIT Centrality Info overload VIT Centrality -0.628
1734 Satisfaction VIT Centrality Info overload VIT Nodes -0.629
1735 Satisfaction VIT Centrality Info overload VIT Centrality -0.629
1736 Education goals VIT Nodes Info overload MessageTag -0.630
1737 Education goals VIT Centrality Info overload MessageTag -0.630
1738 Support CeL Graphs/avatars Functionality VIT Centrality -0.631
1739 Motivation VIT Nodes Info overload VIT Nodes -0.632
1740 Motivation VIT Nodes Info overload VIT Centrality -0.632
1741 Education goals Graphs/avatars Instructions VIT Centrality -0.633
1742 Education goals Graphs/avatars Accessibility VIT Centrality -0.636
1743 Support CeL VIT Nodes VD: Graphics MessageTag -0.638
1744 Support CeL VIT Centrality VD: Graphics MessageTag -0.638
1745 Instructions VIT Centrality VD: Graphics MessageTag -0.641
1746 Education goals Graphs/avatars Accessibility VIT Nodes -0.642
1747 Learnability VIT Nodes VD: Attractiveness VIT Centrality -0.653
1748 Learnability VIT Centrality VD: Attractiveness VIT Centrality -0.653
1749 Originality VIT Nodes Info overload MessageTag -0.655
1750 Functionality MessageTag VD: Attractiveness VIT Nodes -0.657
1751 Support CeL VIT Nodes Fast response Graphs/avatars -0.657
1752 Support CeL VIT Centrality Fast response Graphs/avatars -0.657
1753 Satisfaction VIT Nodes Failure Graphs/avatars -0.665
1754 Instructions VIT Centrality VD: Graphics MessageTag -0.672
1755 Info overload VIT Nodes VD: Attractiveness VIT Nodes -0.676
1756 Info overload VIT Centrality VD: Attractiveness VIT Nodes -0.676
1757 Accessibility VIT Centrality Learnability VIT Nodes -0.687
1758 Accessibility VIT Centrality Learnability VIT Centrality -0.687
1759 Education goals VIT Nodes Info overload Graphs/avatars -0.688
1760 Education goals VIT Centrality Info overload Graphs/avatars -0.688
1761 Accessibility VIT Centrality VD: Graphics MessageTag -0.689
1762 Info overload MessageTag VD: Attractiveness VIT Nodes -0.690
1763 Originality VIT Nodes Info overload Graphs/avatars -0.698
1764 Learnability VIT Nodes VD: Attractiveness VIT Nodes -0.712
1765 Learnability VIT Centrality VD: Attractiveness VIT Nodes -0.712
1766 Info overload MessageTag VD: Attractiveness VIT Centrality -0.719
1767 Accessibility VIT Nodes Learnability VIT Nodes -0.729
1768 Accessibility VIT Nodes Learnability VIT Centrality -0.729
1769 Info overload VIT Nodes VD: Attractiveness VIT Centrality -0.747
1770 Info overload VIT Centrality VD: Attractiveness VIT Centrality -0.747
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 287
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 288
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 289
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 290
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 291
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Glossary
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 292
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Collaborative Learning: The type of learning that takes place when learners work in
groups on the same task using progressive dialogue for co-creativity.
Human Computer Interaction (HCI): The study, planning and design of what
happens when humans interact with computers.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 293
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Learner-Centred Design (LCD): The design that considers the learner/user as the
center of instructional design.
Mirror Neurons: The premotor neurons which fire both when an animal acts and
when the animal observes the same action performed by another (especially of the
same species) animal.
Online Communities: Online social aggregations that emerge when enough people
carry on those public discussions long enough to form relationships.
Online Learning: A planned teaching and learning experience that uses a wide
spectrum of technologies to reach learners at a distance.
Sense of e-Learning Community Index (SeLCI): The index to measure the sense
of belonging in an e-learning community.
Social Network Analysis (SNA): The mapping and measuring of relationships and
flows between people, groups, organizations, computers or other
information/knowledge processing entities. SNA glossary follows:
Closeness: The measurement of the distance between one node and other
node in a network as the number of other nodes divided by the sum of all
distances between the node and all others.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 294
Tools and Evaluation Techniques for Collaborative E-Learning Communities Appendices
Density: The number of actual ties in a network compared to the total amount
of ties that the network can theoretically support.
Ego Network: Consists of a focal node and a set of alter nodes adjacent to or
from the focal node.
Out-degree Centrality: The number of lines that are incident from a node.
Reciprocity: The number of ties that are involved in reciprocal relations relative
to the total number of actual ties.
Usability: A quality attribute that assesses how easy user interfaces are to use.
User-Centered Design (UCD): The design that considers the user into the center of
software design.
Ph.D. Thesis Centre for Interactive Systems Engineering LSBU Page 295